TWM594124U - Artificial intelligence patrol system - Google Patents

Abstract

A smart inspection system suitable for 5G enterprise private network includes: a portable device and an artificial intelligence inspection analysis platform. The portable device includes a camera module, a wireless communication module, a processor and a notification output module. The camera module is used to obtain environmental images. The wireless communication module is used to send environmental information associated with environmental images and receive notification signals. The processor is used to control the wireless communication module to send environmental information. The notification output module is used to output a notification signal. The artificial intelligence inspection and analysis platform is used to receive environmental information, load the 3D model database and route information, perform a comparison procedure based on the environmental information and 3D model database to obtain location information, and generate and send notification signals based on the location information and route information .

Description

Smart inspection system

This creation involves inspection and image recognition, especially a system and method for generating inspection routes in petrochemical plant based on 3D modeling technology and high-speed communication network.

In the petrochemical industry plant, there are complex pipelines, a wide variety of equipment and instruments. In order to maintain the safety of various asset equipment in the plant, the security control center needs to deploy personnel to move along the established route in the plant and conduct inspections. In addition to routinely monitoring device index data, it is necessary to troubleshoot emergencies at any time.

However, for the new inspection personnel, they may not immediately be familiar with the complicated inspection route of the moving line, and may easily break into the dangerous area with high temperature or high pressure gas. It should be noted that the traditional way of setting positioning points in the factory area for route guidance is not suitable for the petrochemical industry. This is because if the electronic sensor installed at the positioning point generates sparks due to failure, it is likely to cause the periphery of the petrochemical pipeline. The chain explosion reaction of leaked flammable gas leads to unpredictable losses.

In view of this, the proposed creation of a kind of wisdom inspection system, the petrochemical industry plant in the region under the fifth generation of mobile communications technology ^{(5 th Generation mobile networks, 5G} ) dedicated spectrum of corporate network deployment environment made of high-speed communication Smart space dynamic line guidance, and realize the intelligent dispatch of labor for each inspector.

An intelligent inspection system described according to an embodiment of the present invention includes: a portable device and an artificial intelligence inspection analysis platform. The portable device includes a camera module, a wireless communication module, a processor and a notification output module. The camera module is used to obtain environmental images. The wireless communication module is used to send environmental information associated with environmental images and receive notification signals. The processor is electrically connected to the camera module and the wireless communication module. The processor is used to control the wireless communication module to send environmental information. The notification output module is electrically connected to the processor. The notification output module is used to output a notification signal. The artificial intelligence inspection and analysis platform communicates with the wireless communication module to receive environmental information. The artificial intelligence inspection analysis platform is used to load the 3D model database and route information. The route information includes the sequence of multiple coordinate points in the 3D model database. The artificial intelligence inspection and analysis platform executes a comparison procedure based on the environmental information and the three-dimensional model database to obtain position information. The artificial intelligence inspection and analysis platform generates and sends notification signals based on location information and route information.

The above description of the content of the disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the creation, and provide a further explanation of the scope of the patent application of the creation.

The following describes in detail the detailed features and advantages of the creation in the embodiments. The content is sufficient for any person skilled in the relevant art to understand and implement the technical content of the creation, and according to the content disclosed in this specification, the scope of patent application and the drawings Anyone who is familiar with related skills can easily understand the purpose and advantages of this creation. The following examples further illustrate the views of this creation in detail, but do not limit the scope of this creation with any views.

Please refer to FIG. 1, which illustrates a block diagram of a smart inspection system 100 described in an embodiment of the present invention. The intelligent inspection system 100 is applicable to the petrochemical industry plant where the high-speed mobile communication network N has been deployed. In one embodiment, the high speed mobile communication network N by the radio access point AP and a plurality of base stations are composed of the wireless access point AP through the fifth-generation mobile communication system technologies ^{(5 th Generation mobile networks, 5G} ) dedicated spectrum of Enterprise private networks are connected to each other.

As shown in FIG. 1, the intelligent inspection system 100 includes: a portable device 3 and an artificial intelligence inspection analysis platform 5. The portable device 3 is communicatively connected to the artificial intelligence inspection and analysis platform 5 through one of the aforementioned wireless access points AP.

As shown in FIG. 1, the portable device 3 includes a camera module 32, a wireless communication module 34, a processor 36 and a notification output module 38. The processor 36 is electrically connected to the camera module 32, the wireless communication module 34, and the notification output module 38. The artificial intelligence inspection and analysis platform 5 communicates with the wireless communication module 34 to receive environmental information. In one embodiment, the artificial intelligence inspection and analysis platform 5 is a server or server cluster, and is communicatively connected to a high-speed mobile communication network N implemented in 5G. In this way, the artificial intelligence inspection and analysis platform 5 can receive the environmental information returned by each portable device 3 under the condition of less delay.

Please refer to FIG. 2, which shows a schematic diagram of the appearance of the portable device 3 described in an embodiment of the present invention. In an embodiment, the portable device 3 is a smart helmet for inspection personnel. Since the camera module 32 is disposed adjacent to the eye position of the inspector wearing the smart helmet-type portable device 3, the inspector can easily collect the factory building he sees through the camera module 32 Internal image information. In other embodiments, the appearance of the portable device 3 may also be smart glasses, a smart phone, a tablet computer, or a personal digital assistant (PDA). This creation does not limit the appearance or hardware type of the portable device 3.

Please refer to FIG. 3, which shows a flowchart of a smart inspection method described in an embodiment of the present invention. The intelligent inspection method is applicable to the aforementioned intelligent inspection system 100. The following further describes the composition and implementation of various components in the smart inspection system 100 in conjunction with the flow of the smart inspection method shown in FIG. 3.

Please refer to step S1, the camera module 32 obtains an environmental image. In detail, in the petrochemical industry plant, the camera module 32 of the portable device 3 worn by the inspector has been activated during the inspection of the inspector, and the camera module 32 can automatically capture the current location of the environment image. For example, the camera module 32 is a high-definition camera to take high-resolution images of the internal environment of a petrochemical plant. In one embodiment, the camera module 32 includes more than two lenses, one of which is set at the position of the forehead, and the other is set at the position of the top of the head. Therefore, the camera module 32 can obtain environmental images with different viewing angles.

In one embodiment, before step S1 (that is, before the camera module 32 of the portable device 3 obtains the environmental image, which is equivalent to the time when the inspector starts the inspection), the following steps are further included: The camera module 32 obtains the starting point image. After obtaining the starting point image, the processor 36 controls the wireless communication module 34 to send the starting point image and the return signal; and the artificial intelligence inspection and analysis platform 5 performs comparison based on the received return signal and the starting point image. The program obtains the confirmation signal and sends the confirmation signal to the portable device 3. In detail, before the inspection personnel start the inspection, they must first arrive at the starting point of the inspection located in the plant. After the inspector arrives at the starting point, the inspector starts the portable device 3 and the camera module 32 shoots the starting point image, and then the inspector performs specific operations on the portable device 3 (for example, in front of the camera of the camera module 32) A specific gesture is provided for the processor 36 to perform image recognition) to send a return signal, which means that the inspector informs the security control center that it is located. After receiving the return signal, the artificial intelligence inspection and analysis platform 5 of the security control center executes the comparison procedure based on a plurality of preset images (preset images of the starting point of inspection) in the 3D model database (or directly A comparison procedure is performed based on all image data in the 3D model database), to thereby confirm whether the starting point image returned by the portable device 3 is correct, and if it is correct, send a confirmation signal to the portable device 3. After completing the above procedure of positioning at the starting point, the inspection personnel can start the inspection of the predetermined route and shoot the environmental image on the inspection route.

In another embodiment, before step S1, the method further includes the following steps: the camera module 32 obtains the starting point image, and the processor 36 compares the starting point image with a plurality of preset images stored therein, when the starting point image and these preset images If one of them matches, the processor 36 controls the wireless communication module 34 to send a preset coordinate point corresponding to the matched preset image. Generally speaking, before starting the inspection, the inspection personnel usually gather at a fixed location in the plant and then set off. Therefore, the processor 36 pre-stores a plurality of environmental images around the starting point of the inspection as preset images, and each preset image corresponds to a preset coordinate point. Therefore, after the processor 36 compares the starting point image with the preset image and confirms that the two match, the coordinate point of the starting position of the inspection personnel can be sent to the artificial intelligence inspection and analysis platform 5 at the security control center, and After the wireless communication module 34 sends the preset coordinate point, the camera module 32 is used to obtain the environmental image. Through the above mechanism, the smart inspection system 100 of an embodiment of the present invention can completely grasp whether each inspection personnel starts inspection from a specified position, and can flexibly decide the inspection route of each inspection personnel, for example: request The inspector moves to the correct starting point of the inspection, or adjusts the following inspection route of the inspector according to the current starting point.

Referring to step S2, the processor 36 controls the wireless communication module 34 to send environmental information. Environmental information is related to environmental images. In one embodiment, the environmental information is an environmental image. In another embodiment, the processor 36 performs an image pre-processing procedure on the environmental image captured by the camera module 32 in advance, thereby reducing the data bandwidth occupied by subsequent environmental information transmission. Regarding the image pre-processing procedure, for example, it may include the following two steps: first, create object features that are not related to the object pose in the environment image, for example, the processor 36 rotates the environment image, and uses local coordinates to image the surface of the pipeline or equipment Encoding of points; Secondly, the region of interest (ROI) is detected in the three-dimensional scene. For example, the processor 36 uses a 3-D cueing method to detect the region of interest in the environmental image. Finally, the environmental information sent by the processor 36 is an image of the region of interest. When multiple inspectors wearing multiple portable devices 3 are inspecting at the same time, the above mechanism can not only shorten the time for image transmission to the security control center, but also improve the overall operation efficiency of the intelligent inspection system 100.

Please refer to step S3, the artificial intelligence inspection and analysis platform 5 loads the three-dimensional model database and receives environmental information. Overall, the artificial intelligence inspection and analysis platform 5 can load the 3D model database and route information, and perform a comparison procedure based on the environmental information and the 3D model database to obtain position information, and generate and send based on the position information and route information Notification signal. In one embodiment, the three-dimensional model database includes one or more Computer Aided Design (CAD) files and multiple coordinate points. For example, the CAD file of the petrochemical industry plant during design can be used to create this 3D model database. The multiple coordinate points include the starting point, end point and relay point of the tour route. Please refer to FIG. 4, which is a schematic diagram showing a plurality of coordinate points a, b, and c on the inspection route. In other embodiments, laser scanning or detailed construction drawings of the petrochemical plant can be used to reversely establish a three-dimensional model database of pipelines and other equipment. For example, a three-dimensional scanning instrument is used to obtain spatial information on the surface of a pipeline or equipment, and a point cloud recognition software is used to convert it into a three-dimensional model with physical meaning. As another example, according to the detailed construction drawings, find the coordinates of the starting point and the reference point of the pipeline, and then use the CAD software system to semi-automatically create a 3D model database.

Please refer to step S4, the artificial intelligence inspection and analysis platform 5 executes the comparison procedure to generate position information. In detail, the artificial intelligence inspection and analysis platform 5 compares the sample images corresponding to the multiple coordinate points of the three-dimensional model database according to the environmental information and determines whether there is a match. In one embodiment, the environmental information is an environmental image or an image of a region of interest, and the comparison program compares the similarity of the two-dimensional images. In another embodiment, the camera module 32 further includes a depth camera or a distance sensor, and the environmental information returned by the portable device 3 further includes a depth camera or a distance sensor. Image depth information, and the comparison program executed by the artificial intelligence inspection and analysis platform 5 compares the similarity of the three-dimensional images. In yet another embodiment, the artificial intelligence inspection and analysis platform 5 receives multiple views of the environment images captured by the portable device 3 at the same time (through multiple photography modules 32 at different installation positions on the portable device 3) Lens) and perform the comparison procedure separately to confirm whether these environmental images belong to the coordinate point range of the same location. Through the above mechanism, the smart inspection system 100 proposed in this creation can improve the accuracy of identifying the location of the inspection personnel with environmental images. It should be added that the artificial intelligence inspection and analysis platform 5 can simultaneously execute the comparison procedure of the two-dimensional images or the identification procedure of the three-dimensional images for the portable devices 3 with different hardware architectures. No restrictions.

Please refer to step S5, the artificial intelligence inspection and analysis platform 5 generates and sends a notification signal based on the location information and route information. The route information includes the sequence of multiple coordinate points in the 3D model database. For example, as shown in FIG. 4, a feasible route information is the patrol route formed by the coordinate points a, b, and c in FIG. In practice, the artificial intelligence inspection analysis platform 5 sends the relevant notification signal to the next coordinate point according to the current location of the inspection personnel. For example, the notification content may include the direction of the next turn and the distance traveled before the next turn.

Referring to step S6, the wireless communication module 34 receives the notification signal and the notification output module 38 outputs the notification signal. The wireless communication module 34 can not only send environmental information associated with environmental images, but also receive notification signals. Please refer to FIG. 1, the notification output module 38 is electrically connected to the processor 36. The notification output module 38 is used to output a notification signal. In one embodiment, the notification signal includes a route image, and the notification output module 38 is a display device, such as the glasses lens on the smart helmet shown in FIG. 2. The display device can present route images. Therefore, the inspection staff can gradually complete the inspection route according to the direction indicated by the route image. In another embodiment, the notification signal includes a voice, and the notification output module 38 is a speaker, and the speaker is used to play the voice. Therefore, the inspection personnel can gradually complete the inspection route according to the voice indicated by the speaker.

In another embodiment, the portable device 3 further includes an inertial measurement unit (IMU) electrically connected to the processor 36. The inertial measurement module is used to obtain mobile information of the portable device 3. And the aforementioned environment image is a first environment image, the environment information is a first environment information, and the position information is a first position information. After step S1, the processor 36 is further used to control the camera module 32 to obtain the first environment image and obtain the second environment image after an interval. The wireless communication module 34 is further used to send second environment information and mobile information associated with the second environment image. The artificial intelligence inspection and analysis platform 5 is further used to receive the second environment information, and perform a comparison procedure based on the second environment information, the three-dimensional model database, the interval time, and the movement information to obtain a second position information. And selectively generate a warning signal according to the first location information, the second location information and the route information. In practice, in order to prevent newly-introduced inspection personnel from getting lost in a complex factory building, the camera module 32 returns the current environmental images at regular intervals for the artificial intelligence inspection analysis platform 5 to analyze whether the inspection personnel are inspected as scheduled Route forward. If the inspection personnel deviates from the predetermined inspection route, the artificial intelligence inspection analysis platform 5 will send a warning signal, and after the portable device 3 receives the warning signal, it will inform the inspection personnel through the notification output module 38, such as broadcasting "The danger zone has been entered, please step back as soon as possible" voice, or a layer or text message with a clear reminder on the display device.

In summary, the smart inspection system proposed in this creation can improve the execution efficiency of inspection and improve the safety of industrial safety. Conduct personnel control and tracking on the site, and collect feedback on inspection information, and monitor and manage the assets and equipment in the petrochemical plant.

Although this creation is disclosed above with the foregoing embodiments, it is not intended to limit this creation. Without departing from the spirit and scope of this creation, all changes and retouching are within the scope of patent protection of this creation. For the protection scope defined by this creation, please refer to the attached patent application scope.

100: Smart inspection system N: High-speed mobile communication network AP: wireless access point 3: portable device 32: Camera module 34: Wireless communication module 36: processor 38: Notification output module 5: Artificial intelligence inspection and analysis platform a, b, c: coordinate points of inspection route S1~S6: Step

FIG. 1 is a block diagram of a smart inspection system according to an embodiment of the present invention. FIG. 2 is a schematic diagram showing the appearance of a portable device according to an embodiment of the present invention. FIG. 3 is a flowchart of a smart inspection method according to an embodiment of the present invention. FIG. 4 is a schematic diagram showing the inspection route of an embodiment of the present creation.

100: Smart inspection system

N: High-speed mobile communication network

AP: wireless access point

3: portable device

32: Camera module

34: Wireless communication module

36: processor

38: Notification output module

5: Artificial intelligence inspection and analysis platform

Claims (6)

A smart patrol inspection system. The smart patrol inspection system includes: a portable device including: a camera module for acquiring an environmental image; and a wireless communication module for sending associations Receiving environmental information and receiving a notification signal in the environmental image; a processor electrically connecting the camera module and the wireless communication module, the processor is used to control the wireless communication module to send the environmental information; and a notification The output module is electrically connected to the processor, and the notification output module is used to output the notification signal; and an artificial intelligence inspection and analysis platform, which is connected to the wireless communication module to receive the environmental information, and the artificial intelligence inspection The analysis platform is used to load a three-dimensional model database and a route information, the route information includes a sequence of coordinate points of the three-dimensional model database, the artificial intelligence inspection analysis platform is based on the environmental information and the three-dimensional model data The library executes a comparison procedure to obtain location information, and generates and sends the notification signal according to the location information and the route information. The requested item wisdom of the inspection system 1 for a high-speed network environment, the network environment includes a plurality of high-speed wireless access point, wireless access point of the plurality of fifth-generation mobile communication technology applications (5 ^th Generation mobile networks, 5G) enterprise private networks with dedicated spectrum are interconnected with each other, and the wireless communication module is connected to the artificial intelligence inspection and analysis platform through one of the wireless access points. The smart inspection system according to claim 1, wherein the notification signal includes a route image, and the notification output module is a display device for displaying the route image. The smart inspection system according to claim 1, wherein the notification signal includes a voice, and the notification output module is a speaker, and the speaker is used to play the voice. The smart inspection system according to claim 1, further comprising: an inertial measurement module electrically connected to the processor, the inertial measurement module is used to obtain mobile information of the portable device; wherein the environmental image Is a first environment image, the environment information is a first environment information, and the position information is a first position information; the processor is further used to control the camera module to obtain the first environment image and pass an interval After that, a second environment image is obtained; the wireless communication module is further used to send a second environment information and the mobile information associated with the second environment image; the artificial intelligence inspection analysis platform is further used to receive the second environment Information, perform the comparison process based on the second environment information, the three-dimensional model database, the interval time, and the movement information to obtain a second position information; and based on the first position information, the second position information, and the The route information selectively generates a warning signal. The intelligent inspection system according to claim 1, wherein the three-dimensional model database includes a computer-aided design file.

Images