TW202324317A - Intelligent drone railway monitoring system and method comprising a drone, a railway state sensing unit, a long-distance wireless communication unit and a monitoring unit, so as to reduce the probability of train accidents - Google Patents

Abstract

The invention discloses an intelligent drone railway monitoring system and method, which includes a drone, a railway state sensing unit, a long-distance wireless communication unit and a monitoring unit. The drone is equipped with a first signal processing module and a flight control module. The railway state sensing unit is attached to the drone and includes an image capture device that can continuously capture the railway images of the drone flying as the railway image along the railway and a global satellite positioning module. The first signal processing module is informationally connected with the railway state sensing unit through the signal transmission unit. The monitoring unit receives railway images and positioning information through the long-distance wireless communication unit, and the monitoring unit includes an image recognition module and a feature database with multiple object feature samples built in, wherein each object feature sample defines an object name. The image recognition module extracts the object features from the railway image, and executes the railway obstacle identification step, so as to input the object features into the feature database to predict the probability that is matched, and to output the information of railway obstacle identification, so that image recognition and artificial intelligence technologies can be used to identify whether the rail is deformed or has obstacles, and can assist the drone to fly along the railway stably. It has the characteristics of high efficiency, low cost and reducing the probability of train accidents.

Description

Intelligent unmanned aerial vehicle railway monitoring system and method

The invention relates to an intelligent UAV railway monitoring system and method, in particular to an UAV railway monitoring technology that can use image recognition and artificial intelligence technology to determine whether the rail is deformed or has obstacles.

Press, according to the report of Freedom Times, from 2012 to 2018, the cumulative number of accidents and the number of casualties have reached 4565 and 827 respectively, with an average of 652.1 accidents and 118.1 casualties per year; The cumulative number of traffic accidents and casualties accounted for 91.81% and 96.37% of the overall railway. The research report pointed out that the United Nations promoted the "Global Plan for the Decade of Action for Road Safety (2011-2020)" (Global Plan for the Decade of Action for Road Safety, 2011-2020), and the international trend of promoting transportation safety has changed from the past Passive post-improvement actions are transformed into active pre-prevention actions; in terms of railways, it is developing towards the establishment of a railway safety management system. However, the report points out that the Railway Bureau handles the engineering construction, supervision and management of railways, mass rapid transit and other railway transportation systems. , has not yet formulated a national railway safety-related plan.

From the above reports, we know that Taiwan’s railway traffic accidents are high, and the international transportation safety trend has changed from passive to active, but there is no practical solution so far. In Taiwan, where the railway utilization rate is extremely high, this situation is very dangerous. Yes, any accident, ranging from damage to property Therefore, this plan proposes this set of equipment, which uses the three functions of this set of equipment to carry out early warning and monitoring of railways, such as autonomous flight along the railway, artificial intelligence railway safety identification, and maintaining flight altitude with terrain changes. During execution, LiDAR is used to set the precise altitude, so that the unmanned aerial vehicle can detect the railway at a stable altitude. When the artificial intelligence identifies any situation on the railway, the monitoring station will immediately receive a notification and transmit the abnormality at the same time. With the location information, the monitoring personnel can clearly know the type of problem and abnormal location of the railway, and take urgent and appropriate measures.

Furthermore, there have been many studies on Unmanned Aerial Vehicle (UAV) at home and abroad. Unmanned Aerial Vehicles can provide exploration, aerial photography, agriculture, disaster relief...etc. Mounting sensing elements and camera lenses, monitors can use its characteristics to monitor the real-time status of the railway clearly and flexibly, and use high-efficiency, low-cost, and high-flexibility equipment to reduce the rate of railway accidents and achieve proactive preventive actions .

As far as we know, prior patents using image recognition or sensing technology to identify railway conditions are as follows:

1. As shown in the new No. M461574 "Obstacle notification and image transmission system for railway level crossings". It is that each level crossing equipment includes an obstacle sensor, a camera device, a first wireless transmission module and a controller, and the controller controls the obstacle sensor to start or stop detection, and controls the camera device to shoot real-time Image, the controller controls the first wireless transmission module to send the real-time image and the level crossing coordinates; the train equipment includes a GPS positioning unit, a second wireless transmission module and a computing module, and the computing module controls the GPS The positioning unit generates a train real-time coordinates; wherein, the calculation module of the train equipment controls the second wireless transmission module to receive the level crossing coordinates sent by the first wireless transmission module of the plurality of level crossing equipment, and the calculation The module compares it with the real-time coordinates of the train to obtain the coordinates of the level crossing closest to the real-time coordinates of the train. Although the train of the patent can receive the real-time image of the closest level crossing; however, the patent does not There is no image recognition technology and the construction of drones, so that it is only possible to identify whether there are obstacles in the real-time image of the railway through the driving of the train with the naked eye, so it is easier to cause image misjudgment due to human negligence or fatigue caused by long-term work, so Causing railway accidents.

2. As shown in the new No. M465319 "Railway Traffic Detecting Foreign Object System". It includes a photographic lens. When the system is activated, the internal lens of the camera will automatically search for the surrounding images of the road conditions and perform image comparison. Once an object enters the screen, the image lens will compare the detected object with the system. Its transmission module, when the image compares the photographic lens to determine the abnormal object, will send a message to the module and then transmit the message, and then the next step is the warning display module, when the image is compared with the photographic lens to determine the abnormality Objects, even when there is a discrepancy, the transmission module will send the message to the device for warning action. Although the patent has the function of image recognition; however, the patent does not have the technical construction of drone inspection, so that a set of photographic lenses and matching image recognition equipment must be installed in each railway section, resulting in excessive cost of image recognition. High, resulting in defects that make it difficult to conduct commercial operations.

It can be seen from the above that these patents are indeed not perfect, and there is still a need for further improvement, and based on the urgent needs of related industries; the reason is that the inventors of the present invention have finally developed A set of the present invention is different from above-mentioned prior art.

The first purpose of the present invention is to provide an intelligent UAV railway monitoring system, which mainly uses image artificial intelligence identification technology to judge whether the rail is deformed or has obstacles, and can assist the UAV to fly along the railway stably, so it has high efficiency , low cost and reduce the probability of train accidents. The technical means adopted to achieve the first objective of the present invention include unmanned aerial vehicle, railway state sensing unit, long-distance wireless communication unit and monitoring unit. UAV device There are first signal processing module and flight control module. The railway state sensing unit is attached to the drone and includes an image capture device and a global satellite positioning module that can continuously capture the railway image of the drone flying along the railway and image it as a railway image, and the first signal processing module It is through the information connection between the signal transmission unit and the railway state sensing unit. The monitoring unit receives railway images and positioning information through the long-distance wireless communication unit. The monitoring unit includes an image recognition module and a feature database with multiple object feature samples built in. Each object feature sample defines an object name. The image recognition module The object features are extracted from the railway image, and the railway obstacle identification step is executed to input the object features into the feature database to predict the probability of matching, and output the information of railway obstacle identification.

The second purpose of the present invention is to provide an intelligent UAV railway monitoring system with the deep learning calculation function of artificial intelligence. The technical means adopted to achieve the second objective of the present invention include unmanned aerial vehicle, railway state sensing unit, long-distance wireless communication unit and monitoring unit. The drone is provided with a first signal processing module and a flight control module. The railway state sensing unit is attached to the drone and includes an image capture device and a global satellite positioning module that can continuously capture the railway image of the drone flying along the railway and image it as a railway image, and the first signal processing module It is through the information connection between the signal transmission unit and the railway state sensing unit. The monitoring unit receives railway images and positioning information through the long-distance wireless communication unit. The monitoring unit includes an image recognition module and a feature database with multiple object feature samples built in. Each object feature sample defines an object name. The image recognition module The object features are extracted from the railway image, and the railway obstacle identification step is executed to input the object features into the feature database to predict the probability of matching, and output the information of railway obstacle identification. Wherein, the image recognition module is a deep learning calculation module, and the execution includes the following steps: (a) The training stage step is to establish at least one deep learning model, and input a huge amount of data into the at least one deep learning model The object feature sample and rail track feature sample are input into the identification parameters of railway obstacles and rail deformation identification, and then the deep learning model is used to test the accuracy of image recognition, and then judge whether the accuracy of image recognition is sufficient. When the judgment result is yes, then Output and store the identification result; when the judgment result is negative, make the deep learning model self-correct learning; and (b) run the prediction stage step The step is to input the real-time captured image of the railway into the deep learning model, and calculate the corresponding features of the object and the features of the rail by the deep learning model, so as to predict the name of the object and whether it is the obstacle. The railway obstacle identification information of the object is obtained, and the deformation identification information of whether the rail is deformed is predicted.

The third purpose of the present invention is to provide an intelligent UAV railway monitoring system with the functions of distinguishing inspection periods and making up for disabled inspections. The technical means adopted to achieve the third objective of the present invention include unmanned aerial vehicle, railway state sensing unit, long-distance wireless communication unit and monitoring unit. The drone is equipped with a first signal processing module and a flight control module. The railway state sensing unit is attached to the drone and includes an image capture device and a global satellite positioning module that can continuously capture the railway image of the drone flying along the railway and image it as a railway image, and the first signal processing module It is through the information connection between the signal transmission unit and the railway state sensing unit. The monitoring unit receives railway images and positioning information through the long-distance wireless communication unit. The monitoring unit includes an image recognition module and a feature database with multiple object feature samples built in. Each object feature sample defines an object name. The image recognition module The object features are extracted from the railway image, and the railway obstacle identification step is executed to input the object features into the feature database to predict the probability of matching, and output the information of railway obstacle identification. Wherein, the railway includes a plurality of railway sections, and each section of the railway is inspected by an unmanned aerial vehicle in a different inspection period, and each unmanned aerial vehicle transmits its own information through the long-distance wireless communication unit. The identification code, the positioning information and the flight path along the railway; the inspection period includes a routine inspection according to the period when the train is not passing and a pre-inspection before each train arrives at the railway section Pre-navigation security inspection, each adjacent second railway section has two drones, when one of the drones fails, the other drone will extend its inspection trip to Covers the section of railway that the disabled drone is responsible for.

The fourth object of the present invention is to provide an intelligent UAV railway monitoring system with the functions of detection and image recognition of slopes, bridge piers or falling tracks near the railway section. The technical means adopted to achieve the fourth objective of the present invention include unmanned aerial vehicles, railway state sensing units, long-distance wireless Communication unit and monitoring unit. The drone is equipped with a first signal processing module and a flight control module. The railway state sensing unit is attached to the drone and includes an image capture device and a global satellite positioning module that can continuously capture the railway image of the drone flying along the railway and image it as a railway image, and the first signal processing module It is through the information connection between the signal transmission unit and the railway state sensing unit. The monitoring unit receives railway images and positioning information through the long-distance wireless communication unit. The monitoring unit includes an image recognition module and a feature database with multiple object feature samples built in. Each object feature sample defines an object name. The image recognition module The object features are extracted from the railway image, and the railway obstacle identification step is executed to input the object features into the feature database to predict the probability of matching, and output the information of railway obstacle identification. Wherein, it further includes at least one slipping sensing unit arranged on the slope, bridge pier or falling road near the dangerous railway section, and the at least one slipping sensing unit is used to sense whether the slope, the bridge pier or the falling road is When the UAV arrives at the dangerous railway section according to the flight path along the railway line, it will receive the sliding sensing information through a short-range wireless communication unit, and pass the long-distance wireless communication unit to receive the sliding sensing information. The communication unit transmits the slipping sensing data to the monitoring unit for interpretation and processing.

The fifth object of the present invention is to provide an intelligent UAV railway monitoring system that uses image recognition technology to correct the flight path along the railway. The technical means adopted to achieve the fifth objective of the present invention include unmanned aerial vehicle, railway state sensing unit, long-distance wireless communication unit and monitoring unit. The drone is equipped with a first signal processing module and a flight control module. The railway state sensing unit is attached to the drone and includes an image capture device and a global satellite positioning module that can continuously capture the railway image of the drone flying along the railway and image it as a railway image, and the first signal processing module It is through the information connection between the signal transmission unit and the railway state sensing unit. The monitoring unit receives railway images and positioning information through the long-distance wireless communication unit. The monitoring unit includes an image recognition module and a feature database with multiple object feature samples built in. Each object feature sample defines an object name. The image recognition module Extract the object features from the railway image, and execute the railway obstacle identification step, so as to input the object features into the feature database to predict the probability of matching, and output the railway obstacle identification data News. Wherein, when the unmanned aerial vehicle flies along the at least one railway section, the image recognition module executes a railway flight path correction step for correcting the flight path along the railway, including the following steps: a Gaussian blur step for Reduce the noise of the railway image; the edge detection step is used to identify the actual edge in the railway image; the image cropping step removes the image of the uninteresting part of the railway image and only keeps the image of the interesting part; Hough transformation The step is to perform Hough transformation on the image of the part of interest to obtain the coordinate feature of the straight line segment; and the least square method calculation step is to perform the least square method calculation on the coordinate feature of the straight line segment, so as to use the calculation result as a correction for the flight path along the railway line basis.

1: Railway

1a: Railway section

10: Drone

11: The first signal processing module

12: Flight control module

13: Temperature sensor

14: Electronic compass

15,22: Global satellite positioning module

16: Animal drive module

17:Proximity warning module

170: smoke device

171: Warning light device

20: Railway state sensing unit

21: Image capture device

23: LiDAR

24: The second signal processing module

30:Long distance wireless communication unit

31,32,33: 5G communication module

40:Monitoring unit

41: Image recognition module

410: Feature database

41a: Deep Learning Calculation Module

411:Deep Learning Models

50: Signal transmission unit

51,52: Zigbee communication module

60:Slip sensing unit

70: Charging station

71: landing platform

80:Information device

Fig. 1 is a functional block schematic diagram of the embodiment of the present invention.

Fig. 2 is a schematic diagram of the implementation of the inspection of the unmanned aerial vehicle of the present invention in the responsible railway section.

Fig. 3 is a schematic diagram of the implementation of the drone inspection on dangerous slopes according to the present invention.

FIG. 4 is a schematic diagram of the process control implementation of the deep learning calculation module in the training phase of the present invention.

FIG. 5 is a schematic diagram of the process control implementation of the deep learning calculation module in the prediction stage of the present invention.

FIG. 6 is an implementation schematic diagram of a specific implementation architecture of the present invention.

Fig. 7 is a schematic diagram of the implementation of the steps of correcting the flight path along the railway according to the present invention.

Fig. 8 is an implementation schematic diagram of the railway obstacle identification process of the present invention.

Fig. 9 is a schematic diagram of the implementation of the monitoring screen display of the monitoring unit of the present invention.

In order to allow your review committee to further understand the overall technical characteristics of the present invention and the technical means to achieve the purpose of the present invention, specific embodiments and accompanying drawings are hereby described in detail:

Please refer to Fig. 1 ~ 3 and shown in Fig. 6, in order to reach the first purpose of the present invention The first embodiment includes a plurality of drones 10, a plurality of railway state sensing units 20, a long-distance wireless communication unit 30 (such as a combination of a 5G communication module and a 5G mobile communication system) and a monitoring unit 40 (such as a computer ; or server). Each UAV 10 includes a first signal processing module 11 and a flight control module 12 preset with a flight path along the railway, and the flight control module 12 can control and drive the UAV 10 along the flight path along the railway. At least one section 1a of the railway 1 flies. Each railway state sensing unit 20 is attached to the drone 10, and each includes an image capture device that can continuously capture the railway image of the drone 10 flying along the railway 1 and image it as a railway image. 21 and a global satellite positioning module 22 for generating positioning information of the location of the UAV 10 . The first signal processing module 11 is connected with the railway status sensing unit 20 via a signal transmission unit 50 (such as a short-distance Zigbee or Bluetooth communication module), and can receive railway images and positioning information. The monitoring unit 40 is set at the ground monitoring station, and can receive the railway image and positioning information transmitted by the first signal processing module 11 through the long-distance wireless communication unit 30. The monitoring unit 40 includes an image recognition module 41 and A built-in feature database 410 with a plurality of object feature samples, each object feature sample defines an object name, the image recognition module 41 is used to extract at least one object feature for each railway image, and perform a railway obstacle identification Step, when performing the railway obstacle identification step, at least one object feature is sequentially input into the feature database 410, so as to predict the coincidence probability of at least one object feature and the object feature sample, and when the coincidence probability is greater than the preset probability, then read out Corresponding object name, and output the corresponding railway obstacle identification information, when the railway obstacle identification information determines that the object will obstruct the railway 1, then output the first warning information including the location information of the object and the railway obstacle identification information for release Give the ground monitoring station a reference for a monitoring personnel and at least one train running on the railway for reference for train driving; then, the monitoring unit 40 queries the location and time point closest to the object through the built-in traffic database The train at the location, then through the long-distance wireless communication unit 30; or other communication means, the first warning information is sent to the information device 80 (such as a smart phone or computer) of the passing train, when the train driver knows that there is an obstacle in the railway 1 , you can make a stop or other response actions, and every time An information device 80 is provided with a 5G communication module 33 information-linked with the long-distance wireless communication unit 30 .

As shown in Figure 1, the railway state sensing unit 20 further includes a laser 23 for sensing the height distance between the UAV 10 and the railway 1 to generate corresponding height data and an image for opening and closing control The acquisition device 21 and the second signal processing module 24 that converts the height data into a height value, the first signal processing module 11 receives the height value transmitted by the second signal processing module 24 through the signal transmission unit 50, And compare it with one of the built-in preset height values, when the height value is higher or lower than the preset height value, then output the control command to correct the height to the flight control module 12 to control the UAV 10 and the railway 1 The distance between them is maintained at a predetermined height. Specifically, the signal transmission unit 50 shown in FIG. 1 is provided with a set of Zigbee communication modules 51, 52 respectively on the UAV 10 and the railway state sensing unit 20, and further includes a temperature sensor for sensing temperature. Detector 13, an electronic compass 14 for sensing the flight direction of the drone, another set of global satellite positioning modules 15 and a set of 5G communication modules 31; in addition, the monitoring unit 40 is also provided with a set of 5G communication The module 32 enables the monitoring unit 40 to communicate with the first signal processing module 15 on the drone 10 through the long-distance wireless communication unit 30 .

More specifically, the feature database 410 has a plurality of rail track feature samples, and each track feature sample defines a deformation identification information. After the railway obstacle identification step, the image recognition module 41 executes a rail deformation identification step, and by A rail feature is extracted from the railway image, and the rail feature is input into the feature database 410 to predict the probability of the rail feature matching the rail feature sample. When the matching probability is greater than the preset probability, the corresponding deformation identification information is read out. When the deformation identification information determines that the rail is deformed, then output the second warning information including the positioning information of the position of the rail deformation and the deformation identification information to be released to the ground monitoring station for reference by a monitoring personnel and for driving on the rail Then, the monitoring unit 40 inquires through the built-in traffic database the train whose location and time point are closest to the location of the object, and then through the long-distance wireless communication unit 30; or other communication methods will The second warning information is transmitted to the train that is about to pass. When the train driver knows that the railway 1 is deformed, he can stop or do other things. response actions.

For a preferred embodiment, please refer to FIG. 1, the present invention further includes an animal repelling module 16 and a proximity warning module 17 arranged on each drone 10, and the proximity warning module 17 includes a smoke device 170 and a warning light device 171. The animal repellent mod16. When the first signal processing module 11 receives the first warning information and the object is identified as an animal, it will drive the flight control module 12 to control the UAV 10 to approach the object, and the animal driving module 16 will send out a sound to drive the animal away and when there is still a train approaching, the first signal processing module 11 starts the smoke device 170 to send a smoke warning signal and starts the warning light device 171 to send a light warning signal, so as to achieve the purpose of warning train driving.

Please refer to the second embodiment shown in Figures 4 to 5, in order to achieve the second embodiment of the second purpose of the present invention, this embodiment includes the overall technical content of the first embodiment, the image recognition module 41 is a deep The learning calculation module 41a comprises the following steps during execution:

(a) The step of the training stage is to establish at least one deep learning model 411, and input a huge amount of object feature samples and rail track feature samples into the at least one deep learning model 411, and input railway obstacle identification parameters and rail deformation identification parameters, Then the deep learning model 411 tests the correct rate of image recognition, and then judges whether the correct rate of image recognition is sufficient, if the judgment result is yes, then output and store the recognition result; when the judgment result is no, then make the deep learning model 411 self Fix learning.

(b) The step of running the prediction stage is to input the real-time captured railway image into the deep learning model 411, and calculate the corresponding object features and rail features by the deep learning model 411, so as to predict the name of the recognized object and whether it is an obstacle The object's railway obstacle identification information, and predict the deformation identification information of whether the rail is deformed.

More specifically, the deep learning model 411 is a training prediction model based on the YOLO deep learning algorithm.

Please refer to shown in Fig. 2, in order to reach the third embodiment of the third purpose of the present invention, this embodiment except comprising the overall technical content of the above-mentioned first embodiment, this railway 1 comprises a plurality of railway sections 1a, each railway In section 1a, an unmanned aerial vehicle 10 is responsible for inspection during a different inspection period, and each unmanned aerial vehicle 10 transmits its own identification code, positioning information and flight path along the railway through the long-distance wireless communication unit 30 . The sub-inspection period includes a routine inspection according to the period when the train is not passing through and a pre-navigation security inspection before each train arrives at the railway section 1a. Each adjacent two railways Section 1a has an arrangement of two drones 10. When one of the drones 10 is disabled, the other drone 10 will extend its inspection trip to cover the area of railway section 1a that the disabled drone 10 is responsible for. .

Please refer to FIG. 2 , a charging station 70 for charging the UAV 10 is set in each railway section 1a, and the charging station is provided with a landing platform 71 for the UAV 10 to land.

Please refer to Fig. 1, shown in 3, in order to reach the 4th embodiment of the 4th object of the present invention, present embodiment includes besides the overall technical content of above-mentioned 1st embodiment, is located near the dangerous railway section 1a The slipping sensing unit 60 of the side slope, bridge pier or falling road (such as the combination of the opposite type infrared sensor and the microprocessor and the short-distance Zigbee or the bluetooth communication module), the slipping sensing unit 60 is used for Sensing whether the side slope, bridge pier or falling road is slipping to generate slipping sensing information, when the UAV 10 arrives at the dangerous railway section 1a according to the flight path along the railway line, then through a signal transmission unit 50 (such as short-distance Zigbee or Bluetooth communication module) to receive the slipping sensing information, and transmit the slipping sensing data to the monitoring unit 40 through the long-distance wireless communication unit 30 for interpretation and processing. When the sliding sensing data is dangerous information, the monitoring unit 40 sends a steering control command to the UAV 10 through the long-distance wireless communication unit 30, so as to drive a lens of the image capture device 21 on the UAV 10 to face the side. Slopes, bridge piers or falling roads, the railway image is captured and imaged as a slope image, bridge pier image or falling road image, and then transmitted to the monitoring and control unit through the long-distance wireless communication unit 30, so that the image recognition module 41 performs image recognition processing To obtain the slip identification information, when the slip When the fall identification information is identified as a dangerous level, a third warning message including the location information of the dangerous railway section 1a and the slide identification information is output.

Please refer to the fifth embodiment shown in FIG. 7, in order to achieve the fifth embodiment of the fifth purpose of the present invention. In addition to including the overall technical content of the first embodiment, when the UAV 10 flies along the railway section 1a, The image recognition module 41 executes a railway flight path correction step for correcting the railway flight path, which includes the following steps:

The Gaussian blur step used to reduce the noise of the railway image.

The edge detection step is used to identify the actual edge in the railway image.

The image cropping step removes the image of the uninteresting part of the railway image, and only keeps the image of the interesting part.

In the Hough transformation step, the image of the part of interest is subjected to Hough transformation to obtain the coordinate feature of the straight line segment.

The least square calculation step is to perform the least square calculation on the coordinate feature of the straight line segment, and use the calculation result as a basis for correcting the flight path along the railway.

In recent years, unmanned aerial vehicles, artificial intelligence, LiDAR 23 and 5G technologies have become increasingly mature. This invention uses the aforementioned technologies and equipment to develop an intelligent UAV railway 1 monitoring system for the railway industry, and a railway state sensing unit. The 5G long-distance wireless communication technology is adopted between the 20 and the monitoring unit 40, so that the data transmission range is not limited by the distance. This is a system with autonomous flight along the railway, artificial intelligence railway safety identification, and flight altitude maintenance with terrain changes. So that monitors can keep abreast of the railway security situation. In addition, the railway state sensing unit 20 is currently designed as an independent plug-in type, which can facilitate maintenance and future function expansion applications.

The overall system architecture of the present invention is shown in Figures 1 and 6. The system can be mainly divided into three parts, namely the railway state sensing unit 20, the UAV flight management system and the monitoring unit 40 of the ground monitoring station. The railway state sensing unit 20 includes a light sensor 23, an image capture device 21, a global satellite positioning module The group 22 (GPS), the second signal processing module 24 (that is, the microcontroller MCU) and the ZigBee communication module 52 are mounted on the UAV 10 in an independent plug-in type, and the microcontroller (Microcontroller Unit, MCU) adopts SoC (System on a Chip) technology has the advantages of miniaturization and multi-function, which can effectively reduce the size of the device. The sensing device receives the data obtained from the components, uses the firmware to write a program, collects and calculates the component data, and integrates and encodes them into a complete data format. The encoded data is the position parameter, and its parameter value is sent to the unmanned flight via ZigBee The vehicle finally unifies and transmits the data back to the monitoring unit 40 by the UAV 10 using the long-distance wireless communication unit 30 (that is, 5G long-distance wireless communication technology) for real-time monitoring of the railway 1 .

In addition, the images observed by the UAV 10 camera are streamed back to the ground monitoring station, and the ground monitoring station is used to identify the railway and predict the extension direction of the railroad track, and finally send the predicted path back to the UAV 10 for flight tasks, so as to achieve For the purpose of railway flight, the concept of route identification is implemented by using the OPENCV package. Since it is necessary to advance along the direction of the railway, it is necessary to use the above-mentioned steps of correcting the flight path along the railway by using the image streamed from the railway track.

The present invention uses YOLOV4 for image recognition, classifies and recognizes obstacles on the railway 1, and displays them on the ground monitoring station with real-time images. The way to find objects is to use the confidence level of all rectangular squares, delete and determine not to include A rectangle of any object. If none of the rectangles in the grid contain objects, the grid is deleted. Since the object may span multiple rectangular blocks at the same time, some overlapping rectangular blocks can be eliminated by using the NMS algorithm. Each graph paper detects a single object. If there are multiple objects in the grid, the one with the highest confidence level is taken as the object in the grid. After repeated execution in this way, the remaining squares are the selected objects. Finally, combined with the selected object and the category of the corresponding grid, the category of the object can be determined. The specific identification process is shown in Figure 8.

The present invention designs an integrated graphic interface, and the software interface uses C# to design C# is an object-oriented high-level programming language based on the .NET framework launched by Microsoft. It writes programs for each object. It has two screens, one is the program interface to plan each object, and the other is the program interface. Designed to be installed and executed on a general computer, the system receives data sent by 5G long-distance wireless communication technology, performs image recognition, analysis, and calculation on the returned image and flight data, and graphically displays it on the instrument interface. Through hardware and The integration of software provides monitors with a clear monitoring screen, which includes information such as electronic maps, real-time images, artificial intelligence recognition results, and flight information. The monitoring screen is shown in Figure 9. The present invention is based on the design of the microcontroller (MCU) in the original plan of the Ministry of Science and Technology to perform measurement, capture calculation and word string comparison and then store it in the embedded solid-state memory, and use short-distance Zigbee, Long-distance 5G mobile communication technology, GPS and microcontroller-encoded flight data are transmitted to the ground receiver and receiving station of the vehicle payload, and the ground monitoring station personnel can use digital instruments to instantly grasp the attitude, height and position of the UAV 10 Wait for flight information to ensure that the aircraft is active on the flight field and avoid collisions between aircraft.

Through the description of the above specific embodiments, the present invention does have the following characteristics:

1. The present invention really uses image recognition and artificial intelligence technology to determine whether the rail is deformed or has obstacles, and can assist the UAV to fly along the rail stably, so it has the characteristics of high efficiency, low cost, and reduced train accident probability. .

2. The present invention does have the deep learning calculation function of artificial intelligence to increase the success rate of image recognition.

3. The present invention does have the functions of distinguishing inspection time periods and making up for disabled inspections, so as to improve the efficiency of railway inspections.

4. The present invention does have the functions of detection and image recognition of slopes, bridge piers or falling tracks near the railway section, so as to improve the safety of trains running on the railway.

5. The present invention does have the ability to use image recognition technology to correct the flight path along the railway The function of the path has achieved the purpose of precisely controlling the flight of the drone.

The above is only a feasible embodiment of the present invention, and is not intended to limit the patent scope of the present invention. Any equivalent implementation of other changes based on the content, characteristics and spirit of the following claims should be Included in the patent scope of the present invention. The structural features of the invention specifically defined in the claims are not found in similar items, and are practical and progressive, and have met the requirements of an invention patent. I file an application in accordance with the law. I would like to ask the Jun Bureau to approve the patent in accordance with the law to maintain this invention. The legitimate rights and interests of the applicant.

1: Railway

1a: Railway section

10: Drone

11: The first signal processing module

12: Flight control module

13: Temperature sensor

14: Electronic compass

15,22: Global satellite positioning module

16: Animal drive module

17:Proximity warning module

170: smoke device

171: Warning light device

20: Railway state sensing unit

21: Image capture device

23: LiDAR

24: The second signal processing module

30:Long distance wireless communication unit

31,32,33: 5G communication module

40:Monitoring unit

41: Image recognition module

410: Feature database

50: Signal transmission unit

51,52: Zigbee communication module

80:Information device

Claims (10)

An intelligent unmanned aerial vehicle railway monitoring system, which includes: At least one unmanned aerial vehicle, which includes a first signal processing module and a flight control module preset with a flight path along the railway, and the flight control module controls and drives the unmanned aerial vehicle along at least one flight path along the railway. flying over at least one section of the railway; At least one railway state sensing unit, which is attached to the at least one unmanned aerial vehicle and each includes an image capture unit that can continuously fly the unmanned aerial vehicle along the at least one railway to perform railway image acquisition and image it as a railway image An acquisition device and a global satellite positioning module for generating positioning information; wherein, the first signal processing module is connected with the at least one railway state sensing unit through a signal transmission unit to receive the railway image and the location information; a long-range wireless communication unit; and A monitoring unit, which is set at a ground monitoring station and receives the railway image and the positioning information transmitted by the first signal processing module through the long-distance wireless communication unit; the monitoring unit includes an image recognition module and an A feature database with a plurality of object feature samples is built in, each object feature sample defines an object name, and the image recognition module is used to extract at least one object feature for each railway image, and execute a railway obstacle identification step , so as to sequentially input the at least one object feature into the feature database to predict the coincidence probability of the at least one object feature and the object feature sample, and when the coincidence probability is greater than a preset probability, the corresponding The name of the object, and output the railway obstacle identification information of the object that matches the object name. When the railway obstacle identification information determines that the object will become an obstacle to the railway, then output the positioning information including the location of the object. and the first warning information of the railway obstacle identification information are issued to the ground monitoring station for reference by a monitoring personnel and at least one train running on the railway for reference by train drivers. The intelligent UAV railway monitoring system as described in claim 1, wherein the at least one railway state sensing unit further includes a method for sensing the height distance between the at least one UAV and the railway A light sensor for generating corresponding height data and a second signal processing module for turning on and off controlling the image capture device and converting the height data into a height value, the first signal processing module passes the signal The transmission unit receives the height value transmitted by the second signal processing module, and compares it with a built-in preset height value, and when the height value is higher or lower than the preset height value, it outputs A control command for height correction is given to the flight control module to control the distance between the at least one drone and the railway to maintain a predetermined height. The intelligent UAV railway monitoring system as described in claim 1, wherein, the feature database has a plurality of rail feature samples, and each rail feature sample defines a deformation identification information. After the railway obstacle identification step, the The image recognition module executes a rail deformation recognition step, and extracts a rail feature from the rail image, so as to input the rail feature into the feature database, so as to predict the coincidence probability of the rail feature and the rail feature sample, when the When the matching probability is greater than a preset probability, the corresponding deformation identification information is read out, and when the deformation identification information determines that the rail is deformed, the positioning information including the position of the deformation of the rail and the second information of the deformation identification information are output. The warning information is issued to the ground monitoring station and the at least one train. In the intelligent UAV railway monitoring system described in claim 3, wherein the image recognition module is a deep learning calculation module, the execution includes the following steps: (a) The training stage step is to establish at least one deep learning model, and input a huge amount of feature samples of the object and rail feature samples into the at least one deep learning model, and input railway obstacle identification parameters and rail deformation identification parameters, and then Test the accuracy of image recognition by the deep learning model, and then judge whether the accuracy of image recognition is sufficient. If the judgment result is yes, the recognition result will be output and stored; if the judgment result is no, the deep learning model will self-correct and learn ;and (b) The step of running the prediction stage is to input the real-time captured railway image into the deep learning model, and calculate the corresponding features of the object and the features of the rail track by the deep learning model, so as to predict the object that recognizes the object Name and identification information of the railway obstacle whether it is the obstacle, and The deformation identification information for predicting whether the rail is deformed. The intelligent UAV railway monitoring system as described in claim 1, wherein the railway includes a plurality of railway sections, and each section of the railway is inspected by an unmanned aerial vehicle in a different inspection period, each of which UAVs use the long-distance wireless communication unit to transmit their respective identification codes, the positioning information and the flight path along the railway line; the divided inspection period includes a routine inspection according to the period when the train is not passing And a pre-flight security inspection before each train arrives at the railway section. Each adjacent railway section has two drones. When one of the drones fails , and the other UAV extends its inspection trip to cover the railway section that the disabled UAV is in charge of. Each railway section is provided with a charging station for charging the UAV. The charging station has a A landing platform for the drone to land on. The intelligent UAV railway monitoring system as described in claim 1, which further includes an animal repelling module and a proximity warning module arranged on the at least one UAV, and the proximity warning module includes a smoke device and A warning light device, when the first signal processing module receives the first warning information and the object is recognized as an animal, it drives the flight control module to control the at least one unmanned aerial vehicle to approach the object, and uses the animal to drive away The module sends out a sound to drive away the animals; and when there is still a train approaching, the first signal processing module activates the smoke device to send a smoke warning signal and activates the warning light device to send a light warning signal. The intelligent unmanned aerial vehicle railway monitoring system as described in claim 1, which further includes at least one slipping sensing unit located on the slope, pier or falling track near the dangerous railway section, and the at least one slipping sensing unit is used Slip sensing information is generated by sensing whether the slope, the bridge pier or the falling road is slipping, and when the UAV arrives at the dangerous railway section according to the flight path along the railway, it is received by a signal transmission unit The slipping sensing information is transmitted to the monitoring unit through the long-distance wireless communication unit for interpretation and processing. The intelligent UAV railway monitoring system as described in claim 7, wherein, when the slipping sensing data is a dangerous message, the monitoring unit sends the UAV to the UAV through the long-distance wireless communication unit Issue a steering control command to drive a lens of the image capture device on the UAV to capture the railway image after the slope, the bridge pier or the falling road and image it as a slope image, bridge pier image or falling road The image is then transmitted to the monitoring and control unit through the long-distance wireless communication unit, so that the image recognition module can perform image recognition processing to obtain the slip identification information. The positioning information of the railway section and the third warning information of the slide identification information. The intelligent UAV railway monitoring system as described in claim 1, wherein, when the UAV flies along the at least one railway section, the image recognition module executes a railway along the railway for correcting the flight path along the railway The flight path correction step includes the following steps: Gaussian blur step, used to reduce the noise of the railway image; an edge detection step for identifying actual edges in the railway image; The image cropping step removes the image of the uninteresting part of the railway image, and only keeps the image of the interesting part; The Hough transformation step is to perform Hough transformation on the image of the part of interest to obtain the coordinate feature of the straight line segment; and The least square calculation step is to perform the least square calculation on the coordinate feature of the straight line segment, and use the calculation result as a basis for correcting the flight path along the railway. A kind of intelligent unmanned aerial vehicle railway monitoring method, it comprises: Provide at least one unmanned aerial vehicle, at least one railway state sensing unit, a long-distance wireless communication unit and a monitoring unit; The at least one unmanned aerial vehicle is provided with a first signal processing module and a flight control module with a preset flight path along the railway, and the flight control module controls and drives the unmanned aerial vehicle along the railway according to the flight path along the railway. fly at least one section of the railway; The at least one railway state sensing unit is attached to the at least one unmanned aerial vehicle and each includes a railway image capture unit that can continuously fly the unmanned aerial vehicle along the railway and image it as a railway image. An image capture device and a global satellite positioning module for generating positioning information; wherein, the first signal processing module is connected with the at least one railway state sensing unit through a signal transmission unit and can receive the railway the image and the location information; and The monitoring unit is set at the ground monitoring station, and the railway image and the positioning information transmitted by the first signal processing module are received through the long-distance wireless communication unit; the monitoring unit includes an image recognition module and an internal A feature database with a plurality of object feature samples is built, each of the object feature samples defines an object name, the image recognition module is used to extract at least one object feature from each of the railway images, and execute a railway obstacle identification step, inputting the at least one object feature into the feature database in order to predict the coincidence probability of the at least one object feature and the object feature sample; when the coincidence probability is greater than the preset probability, the corresponding object is read out name, and output the railway obstacle identification information of the object that matches the name of the object, when the railway obstacle identification information determines that the object will become an obstacle to the railway, then output the positioning information including the location of the object and the The first warning message for railway obstacle identification information.

Images