TW202129229A - Artificial intelligence patrol system and artificial intelligence patrol method - Google Patents

Abstract

An artificial intelligence (AI) patrol system adapted to the 5^th Generation entriprise private network comprises a portable system and an AI patrol analysis platform. The portable device comprises a camera module, a wireless communication module, a processor, and a notification output module. The camera module is configured to obtain an environment image. The wireless communicaton module sends an environment information associated with the environment image and receives a notification signal. The processor is configured to control the wireless communication module to send the environment information. The notification output module is configured to output the notification signal. The AI patrol analysis platform is configured to receive the environment information, load a three-dimensinal model database and a route information, perform a comparison procedure to generate a position information according to the environment information and the three-dimensional model database, generate and send the notification signal according to the position information and the route information.

Description

Smart inspection system and smart inspection method

The invention relates to inspection and image recognition, in particular to a system and method for generating inspection routes in petrochemical industry factories based on three-dimensional modeling technology and high-speed communication network.

In the petrochemical industry plant, there are complex pipelines, a wide variety of equipment and instruments. In order to maintain the safety of various assets and equipment in the plant, the security control center needs to deploy personnel to move in the plant according to the established route and conduct inspections. In addition to routinely monitoring equipment index data, it is also necessary to troubleshoot emergencies at any time.

However, for new patrol personnel, they may not be familiar with the complicated patrol route immediately, and may easily break into dangerous areas with high temperature or high pressure gas. It should be noted that the traditional method of setting up positioning points in the factory area for route guidance is not suitable for the petrochemical industry. This is because if the electronic sensors installed at the positioning points produce sparks due to malfunctions, it is very likely to cause sparks around petrochemical pipelines. The chain explosion reaction of the leaked flammable gas has caused incalculable losses.

Accordingly, the present invention provides a system and wisdom wisdom patrol inspection method, implemented under the dedicated spectrum of the fifth generation of mobile communications technology ^{(5 th Generation mobile networks, 5G} ) corporate network deployment environment made of high-speed communication The intelligent space movement guidance in the petrochemical industry plant area, as well as the realization of the intelligent dispatch of labor for each inspector.

A smart inspection system described according to an embodiment of the present invention includes a portable device and an artificial intelligence inspection and analysis platform. The portable device includes a camera module, a wireless communication module, a processor, and a notification output module. The camera module is used to obtain environmental images. The wireless communication module is used for sending environmental information related to the environmental image and receiving notification signals. The processor is electrically connected to the camera module and the wireless communication module. The processor is used for controlling the wireless communication module to send environmental information. The notification output module is electrically connected to the processor. The notification output module is used to output notification signals. The artificial intelligence inspection and analysis platform is connected to the wireless communication module to receive environmental information. The artificial intelligence inspection and analysis platform is used to load the 3D model database and route information. The route information includes the sequence of multiple coordinate points in the 3D model database. The artificial intelligence inspection and analysis platform executes a comparison procedure based on environmental information and a three-dimensional model database to obtain location information. The artificial intelligence inspection and analysis platform generates and sends notification signals based on location information and route information.

A smart inspection method described according to an embodiment of the present invention includes: a camera module of a portable device obtains an environmental image; a processor of the portable device controls a wireless communication module of the portable device to send an image associated with the environment The environmental information is sent to the artificial intelligence inspection and analysis platform; the artificial intelligence inspection and analysis platform loads the 3D model database; the artificial intelligence inspection and analysis platform executes a comparison procedure based on the environmental information and the 3D model database to obtain location information; artificial intelligence The patrol analysis platform generates and sends notification signals to the portable device based on location information and route information, where the route information includes the sequence of multiple coordinate points in the 3D model database; the wireless communication module of the portable device receives the notification signal; And the notification output module of the portable device outputs the notification signal.

The above description of the disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the present invention, and to provide a further explanation of the scope of the patent application of the present invention.

The detailed features and advantages of the present invention will be described in detail in the following embodiments. The content is sufficient to enable anyone familiar with the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of patent application and the drawings. Anyone who is familiar with relevant skills can easily understand the purpose and advantages of the present invention. The following examples further illustrate the viewpoints of the present invention in detail, but do not limit the scope of the present invention by any viewpoint.

Please refer to FIG. 1, which shows a block diagram of a smart inspection system 100 described in an embodiment of the present invention. The smart inspection system 100 is applicable to petrochemical plants where a high-speed mobile communication network N has been deployed. In one embodiment, the high speed mobile communication network N by the radio access point AP and a plurality of base stations are composed of the wireless access point AP through the fifth-generation mobile communication system technologies ^{(5 th Generation mobile networks, 5G} ) dedicated spectrum of Enterprise private networks are connected to each other.

As shown in FIG. 1, the smart inspection system 100 includes a portable device 3 and an artificial intelligence inspection and analysis platform 5. The portable device 3 is connected to the artificial intelligence inspection and analysis platform 5 through one of the aforementioned wireless access points AP.

As shown in FIG. 1, the portable device 3 includes a camera module 32, a wireless communication module 34, a processor 36 and a notification output module 38. The processor 36 is electrically connected to the camera module 32, the wireless communication module 34 and the notification output module 38. The artificial intelligence inspection and analysis platform 5 is communicatively connected to the wireless communication module 34 to receive environmental information. In one embodiment, the artificial intelligence inspection and analysis platform 5 is a server or a cluster of servers, and is communicatively connected to a high-speed mobile communication network N implemented by 5G. In this way, the artificial intelligence inspection and analysis platform 5 can receive the environmental information returned by each portable device 3 with a relatively small delay.

Please refer to FIG. 2, which shows a schematic diagram of the appearance of the portable device 3 described in an embodiment of the present invention. In one embodiment, the portable device 3 is a smart helmet worn by inspectors. Since the camera module 32 is set close to the eye position of the inspector wearing the portable device 3 of the smart helmet type, the inspector can easily collect the factory buildings he sees through the camera module 32 Internal image information. In other embodiments, the appearance of the portable device 3 may also be smart glasses, smart phones, tablet computers, or personal digital assistants (Personal Digital Assistant, PDA), etc. The present invention does not limit the appearance of the portable device 3 or the type of hardware.

Please refer to FIG. 3, which shows a flowchart of a smart inspection method described in an embodiment of the present invention. The smart inspection method described above is applicable to the aforementioned smart inspection system 100. The following describes the composition and implementation of various components in the smart inspection system 100 in conjunction with the flow of the smart inspection method shown in FIG. 3.

Please refer to step S1, the camera module 32 obtains an environment image. In detail, in the petrochemical industry plant, during the inspection by the inspectors, the camera module 32 in the portable device 3 worn by the inspector has been activated, and the camera module 32 can automatically capture the environment at the current location image. For example, the camera module 32 is a high-definition camera for shooting high-resolution images of the internal environment of the petrochemical plant. In one embodiment, the camera module 32 includes more than two lenses, one of which is set on the forehead and the other on the top of the head. Therefore, the camera module 32 can obtain environmental images with different viewing angles.

In one embodiment, before step S1 (that is, before the camera module 32 of the portable device 3 obtains environmental images, which is equivalent to before the inspector starts the inspection), the following steps are further included: The camera module 32 obtains the starting point image. After obtaining the starting point image, the processor 36 controls the wireless communication module 34 to send the starting point image and the reset signal; and the artificial intelligence inspection and analysis platform 5 performs comparison based on the received reset signal and the starting point image The program obtains the confirmation signal and sends the confirmation signal to the portable device 3. In detail, before the inspection personnel start the inspection, they must first arrive at the starting point of the inspection in the factory. After the inspector arrives at the starting point, the inspector activates the portable device 3 and the camera module 32 captures an image of the starting point, and then the inspector performs specific operations on the portable device 3 (for example, in front of the lens of the camera module 32). A specific gesture is compared for the processor 36 to perform image recognition) to send a reset signal, which means that the inspector informs the security control center that it has been located. After the artificial intelligence inspection and analysis platform 5 of the security control center receives the reset signal, it executes the comparison program (or directly Perform a comparison procedure based on all the image data in the 3D model database) to confirm whether the starting image returned by the portable device 3 is correct, and if it is correct, a confirmation signal is sent to the portable device 3. After completing the above-mentioned procedure of locating at the starting point, the patrol personnel can start the patrol of the established route and shoot the environmental images on the patrol route.

In another embodiment, before step S1, it further includes the following steps: the camera module 32 obtains the starting point image, and the processor 36 compares the starting point image with a plurality of preset images stored by itself, when the starting point image and the preset images If one of them matches, the processor 36 controls the wireless communication module 34 to send a preset coordinate point corresponding to the matched preset image. Generally speaking, the inspection personnel usually gather at a fixed place in the factory before starting the inspection. Therefore, the processor 36 pre-stores a plurality of environmental images around the starting point of the inspection as preset images, and each preset image corresponds to a preset coordinate point. Therefore, after the processor 36 compares the starting point image with the preset image and confirms that the two are consistent, the coordinate point of the starting position of the patrol personnel can be sent to the artificial intelligence patrol analysis platform 5 in the security control center, and After the wireless communication module 34 sends the preset coordinate points, the camera module 32 starts to obtain the environment image. Through the above mechanism, the smart inspection system 100 of an embodiment of the present invention can fully grasp whether each inspector starts the inspection from a specified position, and can flexibly determine the inspection route of each inspector, for example: The inspector moves to the correct starting point of the inspection, or adjusts the next inspection route of the inspector according to the current starting point.

Please refer to step S2, the processor 36 controls the wireless communication module 34 to send environmental information. Environmental information is associated with environmental images. In one embodiment, the environmental information is environmental images. In another embodiment, the processor 36 pre-processes the environmental images captured by the camera module 32 to reduce the data bandwidth occupied by the subsequent environmental information transmission. Regarding the image pre-processing procedure, for example, it may include the following two steps: first, create object features that are not related to the posture of the object in the environment image. For example, the processor 36 rotates the environment image and uses local coordinates to image the surface of the pipeline or equipment. The points are encoded; secondly, a region of interest (Region of Interest, ROI) is detected in the three-dimensional scene. For example, the processor 36 uses a three-dimensional cueing method (3-D cueing) to detect the region of interest in the environmental image. Finally, the environmental information sent by the processor 36 is an image of the region of interest. When multiple inspectors wearing portable devices 3 perform simultaneous inspections, the above-mentioned mechanism can not only shorten the time for image transmission to the security control center, but also improve the overall operation efficiency of the smart inspection system 100.

Please refer to step S3, the artificial intelligence inspection and analysis platform 5 loads the 3D model database and receives environmental information. On the whole, the artificial intelligence inspection and analysis platform 5 can load a 3D model database and route information, and perform a comparison procedure based on the environmental information and the 3D model database to obtain location information, and generate and send based on the location information and route information Notification signal. In one embodiment, the 3D model database includes one or more Computer Aided Design (CAD) files and a plurality of coordinate points. For example, the CAD file of the petrochemical industry plant during the design can be used to create this three-dimensional model database, and the multiple coordinate points include the starting point, the end point and the relay point of the patrol route. Please refer to FIG. 4, which is a schematic diagram showing multiple coordinate points a, b, and c on the inspection route. In other embodiments, laser scanning or detailed construction drawings of a petrochemical plant can be used to reversely build a three-dimensional model database of pipelines and other equipment. For example, use a three-dimensional scanning instrument to obtain the spatial information of the pipeline or equipment surface, and use point cloud recognition software to convert it into a solid model with physical meaning. For another example, find out the coordinates of the start and end points or reference points of the pipeline according to the detailed construction drawings, and then semi-automatically build a 3D model database by the CAD software system.

Please refer to step S4, the artificial intelligence inspection and analysis platform 5 executes a comparison procedure to generate location information. In detail, the artificial intelligence inspection and analysis platform 5 compares the sample images corresponding to multiple coordinate points in the three-dimensional model database according to the environmental information and determines whether there is a match among them. In one embodiment, the environment information is an environment image or an image of a region of interest, and the comparison procedure is to compare the similarity of the two-dimensional image. In another embodiment, the camera module 32 further includes a depth camera or a distance sensor, and the environmental information returned by the portable device 3 further includes information obtained by the depth camera or the distance sensor. The image depth information, and the comparison program executed by the artificial intelligence inspection and analysis platform 5 compares the similarity of the three-dimensional image. In another embodiment, the artificial intelligence inspection and analysis platform 5 receives environmental images of multiple viewing angles captured by the portable device 3 at the same time (through multiple camera modules 32 in different positions on the portable device 3). Lens) and perform the comparison procedure respectively to confirm whether these environmental images belong to the coordinate point range of the same position. Through the above-mentioned mechanism, the smart inspection system 100 proposed in the present invention can improve the accuracy of identifying the position of the inspector based on the environmental image. It should also be added that the artificial intelligence inspection and analysis platform 5 can selectively execute two-dimensional image comparison procedures or three-dimensional image recognition procedures for portable devices 3 with different hardware architectures at the same time. No restrictions.

Please refer to step S5. The artificial intelligence inspection and analysis platform 5 generates and sends a notification signal based on the location information and route information. The route information includes the sequence of multiple coordinate points in the 3D model database. For example, as shown in FIG. 4, a feasible route information is a patrol route composed of coordinate points a, b, and c in FIG. In practice, the artificial intelligence inspection and analysis platform 5 sends relevant notification signals to reach the next coordinate point based on the current position of the inspector. For example, the notification content may include the direction of the next turn and the estimated distance to travel before the next turn.

Please refer to step S6, the wireless communication module 34 receives the notification signal and the notification output module 38 outputs the notification signal. The wireless communication module 34 can not only send environmental information related to the environmental image, but also receive notification signals. Please refer to FIG. 1, the notification output module 38 is electrically connected to the processor 36. The notification output module 38 is used for outputting notification signals. In one embodiment, the notification signal includes a route image, and the notification output module 38 is a display device, such as the glasses lens on the smart helmet shown in FIG. 2. The display device can present route images. Therefore, the inspection personnel can gradually complete the inspection route according to the direction indicated by the route image. In another embodiment, the notification signal includes a voice, and the notification output module 38 is a speaker, and the speaker is used to play the voice. Therefore, the inspection personnel can gradually complete the inspection route according to the voice indicated by the speaker.

In another embodiment, the portable device 3 further includes an Inertial Measurement Unit (IMU) electrically connected to the processor 36. The inertial measurement module is used to obtain the mobile information of the portable device 3. And the aforementioned environmental image is a first environmental image, the environmental information is a first environmental information, and the location information is a first location information. After step S1, the processor 36 is further used to control the camera module 32 to obtain the first environment image and obtain the second environment image after an interval has elapsed. The wireless communication module 34 is further used for sending second environment information and mobile information related to the second environment image. The artificial intelligence inspection and analysis platform 5 is further used to receive the second environment information, and perform a comparison process based on the second environment information, the three-dimensional model database, the interval time, and the movement information to obtain a second location information. And selectively generate a warning signal based on the first location information, the second location information, and the route information. In practice, in order to prevent new patrol personnel from getting lost in the complex plant, the camera module 32 will return the current environmental images at regular intervals for the artificial intelligence patrol analysis platform 5 to analyze whether the patrol personnel are in accordance with the scheduled inspection The route goes forward. If the inspector departs from the scheduled inspection route, the artificial intelligence inspection analysis platform 5 will send a warning signal, and the portable device 3 will notify the inspector through the notification output module 38 after receiving the warning signal, such as playing "You have entered a dangerous area, please go back as soon as possible" voice, or display a layer or text message with obvious reminder function on the display device.

In summary, the smart inspection system proposed by the present invention can improve the execution efficiency of the inspection and improve the safety of industrial safety. Carry out personnel control tracking on the field, and carry out inspection information collection and feedback, and the monitoring and management of assets and equipment in the petrochemical plant.

Although the present invention is disclosed in the foregoing embodiments, it is not intended to limit the present invention. All changes and modifications made without departing from the spirit and scope of the present invention fall within the scope of the patent protection of the present invention. For the scope of protection defined by the present invention, please refer to the attached scope of patent application.

100: Smart inspection system N: High-speed mobile communication network AP: wireless access point 3: portable device 32: camera module 34: wireless communication module 36: processor 38: Notification output module 5: Artificial intelligence inspection and analysis platform a, b, c: inspection route coordinate points S1~S6: steps

FIG. 1 is a block diagram of a smart inspection system according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing the appearance of a portable device according to an embodiment of the present invention. FIG. 3 is a flowchart of a smart inspection method according to an embodiment of the present invention. Fig. 4 is a schematic diagram showing a patrol route according to an embodiment of the present invention.

100: Smart inspection system

N: High-speed mobile communication network

AP: wireless access point

3: portable device

32: camera module

34: wireless communication module

36: processor

38: Notification output module

5: Artificial intelligence inspection and analysis platform

Claims (10)

A smart patrol inspection system. The smart patrol inspection system includes: a portable device, including: a camera module to obtain an environmental image; a wireless communication module to send associated One of environmental information in the environmental image and receiving a notification signal; a processor electrically connected to the camera module and the wireless communication module, and the processor is used to control the wireless communication module to send the environmental information; and a notification The output module is electrically connected to the processor, the notification output module is used to output the notification signal; and an artificial intelligence inspection and analysis platform, which is communicatively connected to the wireless communication module to receive the environmental information, the artificial intelligence inspection The analysis platform is used to load a three-dimensional model database and a route information, the route information includes a sequence of multiple coordinate points in the three-dimensional model database, the artificial intelligence inspection and analysis platform is based on the environmental information and the three-dimensional model data The library executes a comparison process to obtain a location information, and generates and sends the notification signal based on the location information and the route information. The requested item wisdom of the inspection system 1 for a high-speed network environment, the network environment includes a plurality of high-speed wireless access point, wireless access point of the plurality of fifth-generation mobile communication technology applications (5 ^th Generation mobile Networks, 5G) dedicated spectrum enterprise private networks are connected to each other, and the wireless communication module is connected to the artificial intelligence inspection and analysis platform through one of the wireless access points. The smart inspection system according to claim 1, wherein the notification signal includes a route image, the notification output module is a display device, and the display device is used for presenting the route image. The smart inspection system according to claim 1, wherein the notification signal includes a voice, and the notification output module is a speaker, and the speaker is used for playing the voice. The smart inspection system according to claim 1, further comprising: an inertial measurement module electrically connected to the processor, and the inertial measurement module is used to obtain mobile information of the portable device; wherein the environmental image Is a first environment image, the environment information is a first environment information, and the location information is a first location information; the processor is further used to control the camera module to obtain the first environment image and pass an interval time Then obtain a second environment image; the wireless communication module is further used to send a second environment information and the mobile information related to the second environment image; the artificial intelligence inspection and analysis platform is further used to receive the second environment Information, according to the second environment information, the three-dimensional model database, the interval time and the movement information to execute the comparison procedure to obtain a second position information; and according to the first position information, the second position information and the The route information selectively generates a warning signal. The smart inspection system according to claim 1, wherein the three-dimensional model database includes a computer-aided design file. A smart inspection method includes: obtaining an environmental image by a camera module of a portable device; using a processor of the portable device to control a wireless communication module of the portable device to send an associated One of the environmental images of environmental information is sent to an artificial intelligence inspection and analysis platform; the artificial intelligence inspection and analysis platform is used to load a 3D model database; the artificial intelligence inspection and analysis platform is used to execute based on the environmental information and the 3D model database A comparison procedure is used to obtain a location information; the artificial intelligence inspection and analysis platform generates and sends the notification signal to the portable device according to the location information and a route information, wherein the route information includes the data of the 3D model database A sequence of a plurality of coordinate points; the wireless communication module of the portable device receives the notification signal; and a notification output module of the portable device outputs the notification signal. The smart inspection method according to claim 7, wherein the notification signal includes a route image, the notification output module is a display device, and the notification output module of the portable device is used to output the notification signal. The route image is presented by the display device. The smart inspection method according to claim 7, wherein before obtaining the environmental image with the camera module of the portable device, it further comprises: obtaining a point image with the camera module of the portable device; After obtaining the starting point image, use the processor to control the wireless communication module to send the starting point image and a reset signal; and use the artificial intelligence patrol analysis platform to execute the comparison procedure according to the reset signal and the starting point image to obtain A confirmation signal, and sending the confirmation signal to the portable device. The smart inspection method according to claim 7, wherein the processor is further used to store a plurality of preset images, each of the preset images corresponds to one of the preset coordinate points, and the Before the camera module of the portable device obtains the environmental image, it further includes: obtaining a point image with the camera module; comparing the starting point image and each of the preset images with the processor; and when the starting point image In accordance with one of the preset images, the processor controls the wireless communication module to send the preset coordinate point corresponding to the one of the preset images.

Images