Disclosure of Invention
In order to overcome the defects in the prior art, the embodiment of the invention provides a road street lamp inspection system based on an unmanned aerial vehicle, and the invention aims to solve the technical problems that: how to reduce the cost of daily manual inspection and realize the automatic road lamp inspection process.
In order to achieve the purpose, the invention provides the following technical scheme: an unmanned aerial vehicle-based road street lamp inspection system comprises a background server, a personal computer and an unmanned aerial vehicle, wherein the unmanned aerial vehicle is provided with an intelligent control box and a 5G data terminal, the unmanned aerial vehicle is also provided with a pan-tilt camera, and the intelligent control box conducts signals with the background server and the personal computer through the 5G data terminal;
the background server is a high-performance computer, runs an image algorithm, and is responsible for task forwarding, data storage and defect identification; the background server is used as an intermediate end and is responsible for command forwarding, data storage and defect identification, forwarding a flight command of the front end, storing pictures and flight information returned by the intelligent control box, and identifying the street lamp and marking defects through an image processing algorithm;
the personal computer is a display front end, calls the service on the web to plan a routing inspection route, uploads a task after the routing inspection route is planned and monitors an inspection result in real time; the personal computer uses a Web page to perform functional operation, calls a satellite map to autonomously plan a flight route on the personal computer, and can issue the flight route to a background server after the flight route is planned, and the background server determines whether a task is executed according to the position of the unmanned aerial vehicle and the current state of the unmanned aerial vehicle; the personal computer converts the size of the shooting result from the pixel value into the real size based on the laser ranging information transmitted back by the 5G data terminal and the angle of view and distortion parameters of the pan-tilt camera; a user directly calls all the shooting results of the road street lamps through a Web front-end page, checks whether the street lamps are abnormal or not, draws problem components, automatically maps the problem components to a high-precision map and stores the map, and finally automatically generates a routing inspection report;
the operating system of the intelligent control box is set to be ubuntu 16.04, and the dji osdk-ros is used for communicating with the unmanned aerial vehicle;
the 5G data terminal is used for unmanned aerial vehicle real-time video streaming and data transmission;
the cloud deck camera simultaneously acquires high-resolution visible light and infrared thermal imaging photos and carries out laser ranging in real time;
the intelligent control box is communicated with the unmanned aerial vehicle through a serial port, so that the control right of the unmanned aerial vehicle can be obtained, the control right is used for realizing the course line action and the tripod head action of the unmanned aerial vehicle, and the longitude and latitude, the height and the laser ranging value of the aircraft during shooting by the camera are recorded; the intelligent control box is reserved with a network port which can be connected with network equipment to realize networking, and the 5G data terminal is connected to be used as a data link communicated with the background server and used for receiving the routing inspection task and uploading image information.
In a preferred embodiment, the drone comprises an aircraft configured as a macro innovative M210RTK quad rotor drone and a remote controller that uses a thousand know inches network RTK service to provide differential base station data for the aircraft.
In a preferred embodiment, the 5G data terminal is configured to communicate with a "pioneer No. 1" intelligent connector, which can provide 5G access capability for other devices, and is small in size and easy to install on the unmanned aerial vehicle.
In a preferred embodiment, the pan-tilt camera is a jinhualong WK10TIRM three-light pan-tilt camera.
In a preferred embodiment, the intelligent control box is arranged as a berm curie II module, which is based on the university OSDK protocol and is carried on the unmanned aerial vehicle.
In a preferred embodiment, the input end of the unmanned aerial vehicle is electrically connected with the output end of the intelligent control box, and the pan-tilt camera is used for receiving command actions of the unmanned aerial vehicle or the intelligent control box.
In a preferred embodiment, the output end of the 5G data terminal is connected with the intelligent control box and fixed on the unmanned aerial vehicle stand.
The invention has the technical effects and advantages that:
1. the intelligent street lamp inspection system has the advantages that the onboard navigation control is assisted by the video images shot by the onboard camera in real time, a complex flight control system does not need to be developed, the development time is shortened, the maintenance cost is reduced, in addition, the unmanned aerial vehicle can be conveniently controlled through the intelligent control box to complete tasks, networking and data transmission can also be realized, so that data management is formed, the cost of daily manual inspection is reduced based on the street lamp inspection of the unmanned aerial vehicle, the maintenance is convenient, the street lamps can be detected in a long distance by depending on image information obtained by hovering and a pan-tilt camera, inspection personnel do not need to climb each street lamp for maintenance, the inspection efficiency is improved, and the concept of unmanned aerial vehicle + Internet of things is applied, so that the great automatic street lamp inspection process is realized;
2. according to the invention, the background server issues the tasks to the corresponding network addresses from the personal computer to the background server, the 5G data terminal serves as a network service to enable the intelligent control box to receive the uploaded network tasks, so that the remote task receiving is realized, and then the intelligent control box controls the unmanned aerial vehicle to execute the tasks and read the shot pictures and the position information; the background server stores the shot pictures, identifies the street lamps and judges defects through an image algorithm, finally displays the inspection result and generates a report on a personal computer, is used in combination with the intelligent control box to realize the function of the Internet of things, and is combined with an automatic operation flow to realize great convenience and data informatization of road street lamp inspection.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides a road street lamp inspection system based on an unmanned aerial vehicle, which comprises a background server 1, a personal computer 2 and an unmanned aerial vehicle 3, wherein the unmanned aerial vehicle 3 is provided with an intelligent control box 4 and a 5G data terminal 5, the unmanned aerial vehicle 3 is also provided with a pan-tilt camera 6, and the intelligent control box 4 conducts signals with the background server 1 and the personal computer 2 through the 5G data terminal 5;
the background server 1 is a high-performance computer, an image algorithm is operated, and the background server 1 is responsible for task forwarding, data storage and defect identification; the background server 1 serves as an intermediate end and is responsible for command forwarding, data storage and defect identification, forwarding a front-end flight command, storing pictures and flight information returned by the intelligent control box 4, and identifying street lamps and marking defects through an image processing algorithm;
the personal computer 2 is a display front end, calls the service on the web to plan a routing inspection route, uploads a task after the routing inspection route is planned and monitors an inspection result in real time; the personal computer 2 uses a Web page to perform functional operation, calls a satellite map to autonomously plan a flight route on the personal computer, the flight route can be issued to the background server 1 after being planned, and the background server 1 determines whether a task is executed according to the position of the unmanned aerial vehicle 3 and the current state of the unmanned aerial vehicle 3; the personal computer 2 converts the size of the shooting result from the pixel value into the real size based on the laser ranging information sent back by the 5G data terminal 5 and the field angle and distortion parameters of the pan-tilt camera 6; a user directly calls all the shooting results of the road street lamps through a Web front-end page, checks whether the street lamps are abnormal or not, draws problem components, automatically maps the problem components to a high-precision map and stores the map, and finally automatically generates a routing inspection report;
the operating system of the intelligent control box 4 is set as ubuntu 16.04, and the dji osdk-ros is used for communicating with the unmanned aerial vehicle 3;
the 5G data terminal 5 is used for real-time video streaming and data transmission of the unmanned aerial vehicle 3;
the holder camera 6 simultaneously acquires high-resolution visible light and infrared thermal imaging photos and carries out laser ranging in real time;
the intelligent control box 4 is communicated with the unmanned aerial vehicle 3 through a serial port, can acquire the control right of the unmanned aerial vehicle 3, is used for realizing the course line action and the tripod head action of the unmanned aerial vehicle 3, and simultaneously records the longitude and latitude, the height and the laser ranging value of an aircraft during shooting by a camera; the intelligent control box 4 is reserved with a network port which can be connected with network equipment to realize networking, and the connection 5G data terminal 5 is used as a data link for communicating with the background server 1 and is used for receiving the routing inspection task and uploading image information.
The unmanned aerial vehicle 3 comprises an aerial vehicle and a remote controller, the aerial vehicle is set as a Xinjiang innovation M210RTK quadrotor unmanned aerial vehicle 3, the remote controller uses a thousand-inch network RTK service to provide differential base station data for the aerial vehicle, the 5G data terminal 5 is set as a communicated 'Pioneer No. 1' intelligent connector which can provide 5G access capability for other equipment, is small in size and easy to install on the unmanned aerial vehicle 3, the pan-tilt camera 6 is set as a Jinhualong WK10TIRM three-light pan-tilt camera 6, the intelligent control box 4 is set as a Berberi Curie II module which is based on the Xinjiang OSDK protocol and is carried on the unmanned aerial vehicle 3, the input end of the unmanned aerial vehicle 3 is electrically connected with the output end of the intelligent control box 4, the rotor generates power and is autonomously adjusted by the unmanned aerial vehicle 3, and the pan-tilt camera 6 receives the command action of the unmanned aerial, the output end of the 5G data terminal 5 is connected with the intelligent control box 4 and fixed on a foot stand of the unmanned aerial vehicle 3, so that the network communication capacity of the intelligent control box 4 is improved.
As shown in fig. 1 to 3, the embodiment specifically is as follows: the whole cruising process is divided into two major parts of internal industry and external industry, including map drawing, polling process and subsequent maintenance; in the map drawing stage, hardware related equipment comprises an unmanned aerial vehicle 3, a personal computer 2 and a cloud deck camera 6, a remote controller uses a thousand-inch network RTK service to provide differential base station data for the unmanned aerial vehicle 3, a flight route is drawn by the personal computer 2 and uploaded to the unmanned aerial vehicle 3, the unmanned aerial vehicle 3 automatically completes a route flight task, and after the task is completed, orthographic image splicing is performed to draw a flight map; in the inspection process stage, firstly, a user is required to plan a route on a map, and a software platform on the personal computer 2 provides a function of automatically planning the route, namely, the user demarcates an area on the map at the front end of the web, and then the inspection route is divided by a road recognition algorithm; the equipment related to hardware in the routing inspection process comprises an unmanned aerial vehicle 3, a pan-tilt camera 6 and a personal computer 2, wherein the remote controller uses a thousand-inch-finding network RTK service to provide differential base station data for the unmanned aerial vehicle 3; the holder camera 6 can simultaneously acquire visible light and infrared thermal imaging photos with high resolution, can perform laser ranging in real time, and uses the acquired images and laser distance information for street lamp identification and defect marking; the flight route is planned by a personal computer 2, is uploaded to an unmanned aerial vehicle 3 through a background server 1 and an intelligent control box 4, and is automatically finished by the unmanned aerial vehicle 3; in the subsequent maintenance stage, after the flight task is completed, the intelligent control box 4 uploads the task information to the background server 1 through the 5G data terminal 5, a detection person checks a shooting result at the Web front end of the personal computer 2, a problem component is drawn, and the background server 1 automatically maps the problem to a high-precision map and stores the problem; a user can generate a detection report at the Web front end of the personal computer 2, and the report comprises a shooting result, a shooting position, shooting environment information and a problem label, and is delivered to a subsequent unit for follow-up processing in time;
the background server 1 sorts and files data based on task positions, and converts the size of a shooting result from a pixel value into a real size based on laser ranging information and the field angle and distortion matrix parameters of the pan-tilt camera 6;
after data uploading and processing are finished, a user can find an executed detection task at the Web front end of the personal computer 2, pictures are distributed on a schematic navigation line according to shooting positions, the user can check the corresponding condition of visible light and infrared pictures shot at each position, if the street lamp identified in the pictures is abnormal, the user can circle a problem area on the infrared pictures to obtain problem area information, and the problem reason is determined by comparing the problem area information with the visible light pictures; all labels are stored in the background server 1 for storage; finally, after the problem area is repaired, the user can carry the field equipment to the site again, call the previous task at the front end of the web, select the marked waypoint for rechecking, and realize the closed loop of the detection operation;
as shown in fig. 4, the embodiment specifically includes: the specific flow of the inspection process comprises the steps that firstly, a tester carries an unmanned aerial vehicle 3 to an operation site, the unmanned aerial vehicle 3 is assembled and is started for self-inspection, the condition that each state of the unmanned aerial vehicle 3 is normal and RTK is successfully started is ensured, an onboard intelligent control box 4 is a small computer, a course task of a background server 1 is received through a network, then the unmanned aerial vehicle 3 is controlled to take off, the angle of a holder is adjusted and a picture is taken, the unmanned aerial vehicle returns to a flying point after the task is completed, finally, a picture taking result and position information are uploaded to data storage of the background server 1, the background server 1 runs an image algorithm to identify a street lamp and a defect label, identifies a convolution edge detection algorithm combining deep learning and OpenCV, circles out a street lamp frame;
as shown in fig. 5, the embodiment specifically includes: the work flow of the intelligent control box 4 comprises server task acquisition, unmanned aerial vehicle 3 real-Time position acquisition, holder attitude angle and picture reading, and then flight control, video streaming and data uploading are performed, the task state is from the background server 1, the unmanned aerial vehicle 3 real-Time position, holder attitude angle and picture are read from the unmanned aerial vehicle 3 communication interface, the control is position and attitude control in dji osdk-ros, the video streaming is based on RTMP (real Time Messaging protocol), and the data is based on a message protocol of a publishing/subscribing paradigm.
The points to be finally explained are: first, in the description of the present application, it should be noted that, unless otherwise specified and limited, the terms "mounted," "connected," and "connected" should be understood broadly, and may be a mechanical connection or an electrical connection, or a communication between two elements, and may be a direct connection, and "upper," "lower," "left," and "right" are only used to indicate a relative positional relationship, and when the absolute position of the object to be described is changed, the relative positional relationship may be changed;
secondly, the method comprises the following steps: in the drawings of the disclosed embodiments of the invention, only the structures related to the disclosed embodiments are referred to, other structures can refer to common designs, and the same embodiment and different embodiments of the invention can be combined with each other without conflict;
and finally: the above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that are within the spirit and principle of the present invention are intended to be included in the scope of the present invention.