CN114550107B

CN114550107B - Bridge linkage intelligent inspection method and system based on unmanned aerial vehicle cluster and cloud platform

Info

Publication number: CN114550107B
Application number: CN202210441029.8A
Authority: CN
Inventors: 杨翰翔; 赖晓俊
Original assignee: Shenzhen Lianhe Intelligent Technology Co ltd
Current assignee: Shenzhen Lianhe Intelligent Technology Co ltd
Priority date: 2022-04-26
Filing date: 2022-04-26
Publication date: 2022-07-26
Anticipated expiration: 2042-04-26
Also published as: CN114550107A

Abstract

The utility model provides a bridge linkage intelligence inspection method based on unmanned aerial vehicle cluster, system and cloud platform, bridge structure distribution information in the bridge control inspection image can be drawed, the unusual matching record between image format category and the bridge control inspection angle is the unmanned aerial vehicle selection basis of a plurality of dimensions, confirm whether the unmanned aerial vehicle of inspecting to be selected is the unmanned aerial vehicle of inspecting to be allocated that can allocate, treat to allocate the in-process that unmanned aerial vehicle carries out unmanned aerial vehicle and patrols and trunked the linkage allocation analysis of allocating of inspecting to be managed, can ensure diversified and the richness of linkage allocation analysis basis, thereby improve to a certain extent and treat to allocate the cluster of patrolling and examining unmanned aerial vehicle and patrol and allocate the rate of accuracy and credibility, can utilize treat to allocate the unmanned aerial vehicle and patrol and examine to appointed bridge monitoring area and carry out comprehensive and comprehensive patrolling.

Description

Bridge linkage intelligent inspection method and system based on unmanned aerial vehicle cluster and cloud platform

Technical Field

The embodiment of the application relates to the technical field of image processing and unmanned aerial vehicles, in particular to a bridge linkage intelligent inspection method and system based on an unmanned aerial vehicle cluster and a cloud platform.

Background

Unmanned Aerial Vehicles (UAVs) are Unmanned aircraft that are operated by radio remote control devices and self-contained program control devices, or are operated autonomously, either completely or intermittently, by an onboard computer.

With the development of unmanned aerial vehicle technology, the current unmanned aerial vehicle is applied to the fields of aerial photography, agriculture, plant protection, miniature self-timer, express transportation, disaster relief, wild animal observation, infectious disease monitoring, surveying and mapping, news reporting, power inspection, disaster relief, movie and television shooting and the like.

The combination of the image processing technology and the computer vision technology greatly improves the image processing efficiency. By combining the unmanned aerial vehicle with the image processing technology, the further high-speed development of the unmanned aerial vehicle can be promoted to a greater extent. In order to more efficiently realize the inspection work of the bridge, an unmanned aerial vehicle and an image processing technology are often combined; however, the related unmanned aerial vehicle bridge safety inspection technology has the problem of low comprehensive degree.

Disclosure of Invention

In view of this, the embodiment of the application provides a bridge linkage intelligent inspection method and system based on an unmanned aerial vehicle cluster, and a cloud platform.

The embodiment of the application provides a bridge linkage intelligent inspection method based on unmanned aerial vehicle cluster, is applied to bridge linkage intelligent inspection cloud platform among the bridge linkage intelligent inspection system, bridge linkage intelligent inspection cloud platform with unmanned aerial vehicle communication connection patrols and examines among the bridge linkage intelligent inspection system, the method includes:

collecting at least one frame of bridge monitoring inspection image of the inspection unmanned aerial vehicle to be selected;

identifying each frame of bridge monitoring inspection image in the at least one frame of bridge monitoring inspection image, and respectively obtaining at least two of corresponding bridge structure distribution information, image format types and bridge monitoring inspection angles;

if at least two of the bridge monitoring inspection angle, the bridge structure distribution information and the image format category are in abnormal matching, obtaining first inspection influence information corresponding to each frame of bridge monitoring inspection image from an abnormal matching record which is deployed in advance;

determining an unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image;

and determining the inspection unmanned aerial vehicle to be allocated according to the obtained unmanned aerial vehicle selection result.

Alternatively, after receiving at least one frame of bridge monitoring inspection image of the to-be-selected inspection unmanned aerial vehicle, the method further includes:

judging whether the at least one frame of bridge monitoring inspection image contains significant content, wherein the significant content comprises: bridge structure key features or bridge detection equipment information;

if each frame of bridge monitoring inspection image contains the significance content, second inspection influence information corresponding to each frame of bridge monitoring inspection image is obtained from a relational database which is deployed in advance so as to determine at least one second inspection influence information corresponding to at least one frame of bridge monitoring inspection image;

correspondingly, the determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on the at least one first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image includes:

and determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to the at least one first inspection influence information and the at least one second inspection influence information.

Alternatively, after receiving at least one frame of bridge monitoring inspection image of the inspection unmanned aerial vehicle to be selected, the method further includes:

receiving at least one feedback image fed back by aiming at the at least one frame of bridge monitoring inspection image; the bridge linkage intelligent inspection cloud platform receives the at least one frame of bridge monitoring inspection image and then processes the at least one frame of bridge monitoring inspection image to obtain at least one feedback image;

identifying the at least one feedback image to obtain unmanned aerial vehicle selected reference information corresponding to each feedback image;

if the unmanned aerial vehicle selects the reference information to indicate that data to be subjected to safety monitoring inspection is not inquired, obtaining third inspection influence degree information corresponding to each frame of bridge monitoring inspection image from an error feedback list deployed in advance so as to determine at least one third inspection influence degree information corresponding to at least one frame of bridge monitoring inspection image;

determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to the at least one first inspection influence information and the at least one third inspection influence information;

or determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to the at least one first inspection influence information, the at least one second inspection influence information and the at least one third inspection influence information;

wherein the method further comprises:

identifying each frame of bridge monitoring inspection image in the at least one frame of bridge monitoring inspection image, and respectively obtaining uniquely bound bridge monitoring inspection variables;

calculating the number of the bridge monitoring inspection variables corresponding to each frame of bridge monitoring inspection image;

and determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to at least one of the at least one second inspection influence information and the at least one third inspection influence information, the at least one first inspection influence information and the number of the bridge monitoring inspection variables.

Alternatively, the determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on the at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image includes:

and if first target inspection influence information exceeding a first flight interference evaluation threshold exists in at least one first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image, determining that the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the first target inspection influence information is an unmanned aerial vehicle filtering result.

if second target patrol inspection influence information which does not exceed a first flight interference evaluation threshold exists in at least one first patrol inspection influence information corresponding to the at least one frame of bridge monitoring patrol inspection image, fusing the second target patrol inspection influence information to obtain first overall patrol inspection influence information;

if the first overall inspection influence degree information does not exceed a first overall flight interference evaluation threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second target inspection influence degree information is an unmanned aerial vehicle allocation result;

and if the first integrity inspection influence degree information exceeds a first global flight interference evaluation threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second target inspection influence degree information is an unmanned aerial vehicle filtering result.

Alternatively, the first inspection influence information corresponding to each frame of bridge monitoring inspection image comprises: first and second local influence degree information;

if abnormal matching exists among at least two of the bridge monitoring inspection angle, the bridge structure distribution information and the image format type, first inspection influence degree information corresponding to each frame of bridge monitoring inspection image is obtained from an abnormal matching record which is deployed in advance, and the method comprises the following steps:

if the bridge monitoring inspection angle is in abnormal matching with the image format type, obtaining first local influence degree information corresponding to each frame of bridge monitoring inspection image from a first previously deployed abnormal matching record;

or if the bridge structure distribution information and the image format category have abnormal matching, obtaining second local influence information corresponding to each frame of bridge monitoring inspection image from a second previously deployed abnormal matching record;

if there is an abnormal match between the bridge monitoring inspection angle and the image format type, obtaining first local influence degree information corresponding to each frame of bridge monitoring inspection image from a first previously deployed abnormal matching record, including:

if the image format type and the bridge monitoring inspection angle have a first target abnormal matching record which is matched with a first previously deployed abnormal matching record, indicating that the first local influence degree information corresponding to the first target abnormal matching record is obtained from the previously deployed abnormal matching record;

if abnormal matching exists between the bridge structure distribution information and the image format types, second local influence degree information corresponding to each frame of bridge monitoring inspection image is obtained from a second previously deployed abnormal matching record, and the method comprises the following steps:

if the bridge structure distribution information and the image format type have a second target abnormal matching record which is matched with a second previously deployed abnormal matching record, indicating that second local influence degree information corresponding to the second target abnormal matching record is obtained from the previously deployed abnormal matching record;

if the bridge monitoring inspection angle is in abnormal matching with the image format type, after first local influence degree information corresponding to each frame of bridge monitoring inspection image is obtained from a first previously deployed abnormal matching record, the method further comprises the following steps:

calculating a number of categories of a first restrictive image format category corresponding to the first local influence magnitude information;

if the number of the types of the first restrictive image format type exceeds a first set type threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the first restrictive image format type is an unmanned aerial vehicle filtering result;

wherein, if there is an abnormal match between the bridge structure distribution information and the image format category, after obtaining second local influence degree information corresponding to each frame of the bridge monitoring inspection image from a second previously deployed abnormal matching record, the method further comprises:

calculating a number of categories of a second restrictive image format category corresponding to the second local influence magnitude information;

and if the number of the types of the second restrictive image format type exceeds a second set type threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second restrictive image format type is an unmanned aerial vehicle filtering result.

Alternatively, after determining the drone selection result of the at least one frame of bridge surveillance inspection image, the method further includes:

if the unmanned aerial vehicle selected result of the at least one frame of bridge monitoring inspection image is an unmanned aerial vehicle filtering result, deleting the at least one frame of bridge monitoring inspection image;

or if the unmanned aerial vehicle selected result of the at least one frame of bridge monitoring inspection image is an unmanned aerial vehicle filtering result and at least one feedback image fed back by aiming at the at least one frame of bridge monitoring inspection image is received, deleting the at least one feedback image;

or if the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image is an unmanned aerial vehicle allocation result and at least one feedback image fed back by aiming at the at least one frame of bridge monitoring inspection image is received, saving the at least one feedback image.

Alternatively, the determining to-be-allocated inspection unmanned aerial vehicle according to the obtained unmanned aerial vehicle selection result includes:

determining the unmanned aerial vehicle to be selected with the unmanned aerial vehicle selection result as the unmanned aerial vehicle to be allocated;

determining a bridge inspection flight path of an unmanned aerial vehicle cluster consisting of the to-be-allocated inspection unmanned aerial vehicles according to the position distribution information of each to-be-allocated inspection unmanned aerial vehicle;

wherein, according to every wait to allocate the position distribution information of patrolling and examining unmanned aerial vehicle, confirm by wait to allocate the bridge of patrolling and examining the unmanned aerial vehicle cluster that unmanned aerial vehicle constitutes and patrol and examine the flight route, include:

acquiring the space three-dimensional state and flight attitude information of each to-be-allocated inspection unmanned aerial vehicle;

determining a correlation coefficient between each piece of flight attitude information under the static path label of each to-be-allocated patrol unmanned aerial vehicle and each piece of flight attitude information under the dynamic path label of each to-be-allocated patrol unmanned aerial vehicle according to flight attitude information under the dynamic path labels of the plurality of to-be-allocated patrol unmanned aerial vehicles and state description thereof on the premise that each to-be-allocated patrol unmanned aerial vehicle contains a dynamic path label according to the spatial three-dimensional state, and migrating the flight attitude information under the static path label of each to-be-allocated patrol unmanned aerial vehicle, which is similar to the flight attitude information under the dynamic path label, to a corresponding dynamic path label;

under the premise that a plurality of flight attitude information is contained under the current static path label of each to-be-allocated inspection unmanned aerial vehicle, determining a correlation coefficient among the flight attitude information under the current static path label of each to-be-allocated inspection unmanned aerial vehicle according to the flight attitude information under the dynamic path labels of the to-be-allocated inspection unmanned aerial vehicles and the state description of the flight attitude information, and grouping the flight attitude information under the current static path label according to the correlation coefficient among the flight attitude information;

setting dynamic path label description for each group of flight attitude information obtained by grouping according to the flight attitude information and the state description under the dynamic path labels of the plurality of to-be-allocated inspection unmanned aerial vehicles, and migrating each group of flight attitude information to the dynamic path label represented by the dynamic path label description;

and determining each local flight path of the unmanned aerial vehicle to be allocated according to the flight attitude information under the dynamic path label, and performing global fusion on the local flight paths to obtain a bridge inspection flight path of an unmanned aerial vehicle cluster formed by the unmanned aerial vehicles to be allocated.

The embodiment of the application also provides a bridge linkage intelligent inspection cloud platform, which comprises a processor, a network module and a memory; the processor and the memory communicate through the network module, and the processor reads the computer program from the memory and operates to perform the above-mentioned method.

An embodiment of the present application further provides a computer storage medium, where a computer program is stored, and the computer program implements the foregoing method when running.

Compared with the prior art, the bridge linkage intelligent inspection method, system and cloud platform based on the unmanned aerial vehicle cluster have the following technical effects: bridge structure distribution information in the bridge control inspection image can be extracted, abnormal matching records between image format categories and bridge control inspection angles are used as unmanned aerial vehicle selection bases of multiple dimensions, whether the unmanned aerial vehicle to be selected is to be allocated to inspect is determined, the unmanned aerial vehicle to be allocated is subjected to unmanned aerial vehicle inspection cluster linkage allocation analysis, diversification and richness of linkage allocation analysis bases can be ensured, the cluster inspection linkage allocation accuracy and reliability of the unmanned aerial vehicle to be allocated are improved to a certain degree, and the unmanned aerial vehicle to be allocated can be utilized to comprehensively and comprehensively inspect a designated bridge monitoring area.

In the following description, other features will be set forth in part. These features will be in part apparent to those of ordinary skill in the art upon examination of the following and the accompanying drawings or may be learned by production or use. The features of the present application may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities and combinations particularly pointed out in the detailed examples that follow.

Drawings

To more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.

FIG. 1 is a schematic block diagram of a bridge linkage intelligent patrol inspection cloud platform provided by the embodiment of the application.

Fig. 2 is a flowchart of a bridge linkage intelligent inspection method based on an unmanned aerial vehicle cluster provided in the embodiment of the present application.

Fig. 3 is a communication architecture block diagram of the bridge linkage intelligent inspection system provided in the embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.

Thus, the following detailed description of the embodiments of the present application, as presented in the figures, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined or explained in subsequent figures.

Fig. 1 shows a block schematic diagram of a bridge linkage intelligent inspection cloud platform 10 provided by an embodiment of the present application. Cloud platform 10 is patrolled and examined to bridge linkage intelligence in the embodiment of this application can be for having data storage, transmission, processing function's server, as shown in fig. 1, cloud platform 10 is patrolled and examined to bridge linkage intelligence includes: memory 11, processor 12, network module 13 and bridge linkage intelligent inspection device 20 based on unmanned aerial vehicle cluster.

The memory 11, the processor 12 and the network module 13 are electrically connected directly or indirectly to realize data transmission or interaction. For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The storage 11 stores the bridge linkage intelligent inspection device 20 based on the unmanned aerial vehicle cluster, the bridge linkage intelligent inspection device 20 based on the unmanned aerial vehicle cluster includes at least one software functional module which can be stored in the storage 11 in a form of software or firmware (firmware), and the processor 12 executes various functional applications and data processing by operating software programs and modules stored in the storage 11, such as the bridge linkage intelligent inspection device 20 based on the unmanned aerial vehicle cluster in the embodiment of the present application, so as to implement the bridge linkage intelligent inspection method based on the unmanned aerial vehicle cluster in the embodiment of the present application.

The Memory 11 may be, but is not limited to, a Random Access Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like. The memory 11 is used for storing a program, and the processor 12 executes the program after receiving the execution instruction.

The processor 12 may be an integrated circuit chip having data processing capabilities. The Processor 12 may be a general-purpose Processor including a Central Processing Unit (CPU), a Network Processor (NP), and the like. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The network module 13 is used for establishing communication connection between the bridge linkage intelligent patrol inspection cloud platform 10 and other communication terminal devices through a network, and realizing receiving and transmitting operations of network signals and data. The network signal may include a wireless signal or a wired signal.

It is to be understood that the configuration shown in fig. 1 is merely illustrative and that the bridge linkage smart inspection cloud platform 10 may include more or fewer components than shown in fig. 1 or may have a different configuration than shown in fig. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.

Fig. 2 shows a flow chart of the bridge linkage intelligent inspection based on the unmanned aerial vehicle cluster provided by the embodiment of the application. The method steps defined by the related process of the method are applied to the bridge linkage intelligent inspection cloud platform 10, and the method comprises the following steps of S21-S25.

The method is applied to the bridge linkage intelligent inspection cloud platform in the bridge linkage intelligent inspection system, the bridge linkage intelligent inspection cloud platform is in communication connection with the inspection unmanned aerial vehicle in the bridge linkage intelligent inspection system, the inspection unmanned aerial vehicle to be selected is one or more of the unmanned aerial vehicles in communication connection with the bridge linkage intelligent inspection cloud platform, and the method is not limited in the application.

And step S21, collecting at least one frame of bridge monitoring inspection image of the inspection unmanned aerial vehicle to be selected.

For example, the corresponding bridge monitoring inspection image shot by the inspection unmanned aerial vehicle to be selected aiming at the designated bridge monitoring area can be obtained before the unmanned aerial vehicle cluster combination is carried out, and the bridge monitoring inspection image is used for judging whether the inspection unmanned aerial vehicle to be selected accords with the unmanned aerial vehicle cluster combination condition.

And S22, identifying each frame of bridge monitoring inspection image in the at least one frame of bridge monitoring inspection image, and respectively obtaining at least two of corresponding bridge structure distribution information, image format types and bridge monitoring inspection angles.

In this application embodiment, bridge construction distribution information is used for the relative spatial position relation between the different positions of sign bridge, and image format classification is used for patrolling and examining the image to the bridge control and distinguishes, and the angle can be understood as the angle that the image was shot to the bridge control angle of patrolling and examining, and the bridge control angle of patrolling and examining can the edgewise reflection treat patrolling and examining unmanned aerial vehicle's flight gesture.

And step S23, if abnormal matching exists between at least two of the bridge monitoring inspection angle, the bridge structure distribution information and the image format type, obtaining first inspection influence information corresponding to each frame of bridge monitoring inspection image from the previously deployed abnormal matching records.

It can be understood that, there is conflict or incompatible condition between the information of the relevant dimension of unusual matching characterization bridge control patrol and examine image, under this kind of condition, probably need to patrol and examine the further analysis of influence in coordination to avoid patrolling and examining to have flight conflict or flight interference between the unmanned aerial vehicle, can develop smoothly in order to guarantee the work of patrolling and examining of unmanned aerial vehicle cluster.

Furthermore, the abnormal matching records are used for representing the matching conditions among the information of the relevant dimensionalities of the bridge monitoring inspection images, the first inspection influence degree information with quantitative analysis value can be obtained through the abnormal matching records, and the first inspection influence degree information is used for representing the quantitative values of various performance evaluations of the inspection unmanned aerial vehicle in the unmanned aerial vehicle cluster forming process, corresponding to the bridge monitoring inspection images.

In some possible embodiments, the first patrol influence degree information corresponding to each frame of bridge monitoring patrol image includes: first and second local influence volume information. Based on this, if there is an abnormal match between at least two of the bridge monitoring inspection angle, the bridge structure distribution information, and the image format type, which is described in step S23, obtaining first inspection influence information corresponding to each frame of bridge monitoring inspection image from the previously deployed abnormal matching records may include the following step 231 or step 232.

And 231, if the bridge monitoring inspection angle is in abnormal matching with the image format type, obtaining first local influence degree information corresponding to each frame of bridge monitoring inspection image from a first previously deployed abnormal matching record.

Further, if there is an abnormal match between the bridge monitoring inspection angle and the image format type, obtaining first local influence degree information corresponding to each frame of bridge monitoring inspection image from a first previously deployed abnormal matching record, including: and if the image format type and the bridge monitoring inspection angle have a first target abnormal matching record which is matched with a first previously deployed abnormal matching record, indicating that the first local influence degree information corresponding to the first target abnormal matching record is obtained from the previously deployed abnormal matching record.

In some optional embodiments, if there is an abnormal match between the bridge monitoring inspection angle and the image format type, after obtaining first local influence degree information corresponding to each frame of the bridge monitoring inspection image from a first previously deployed abnormal matching record, the method further includes: calculating a number of categories of a first restrictive image format category corresponding to the first local influence magnitude information; and if the number of the types of the first restrictive image format type exceeds a first set type threshold value, the unmanned aerial vehicle selection result of the bridge monitoring patrol inspection image corresponding to the first restrictive image format type is an unmanned aerial vehicle filtering result.

In this application embodiment, unmanned aerial vehicle filter result can be understood as not regarding the unmanned aerial vehicle that patrols and examines to be selected as one in the unmanned aerial vehicle cluster, also reflects the correlation performance (such as flight attitude, flight circuit) that the unmanned aerial vehicle that patrols and examines to be selected from the side and is difficult to satisfy and patrols and examines the cooperation requirement.

Step 232, if an abnormal match exists between the bridge structure distribution information and the image format type, obtaining second local influence degree information corresponding to each frame of bridge monitoring inspection image from a second previously deployed abnormal matching record.

Further, if there is an abnormal match between the bridge structure distribution information and the image format type, obtaining second local influence information corresponding to each frame of the bridge monitoring inspection image from a second previously deployed abnormal matching record, including: and if the bridge structure distribution information and the image format type have a second target abnormal matching record matched with a second previously deployed abnormal matching record, indicating that second local influence degree information corresponding to the second target abnormal matching record is obtained from the previously deployed abnormal matching record.

In some optional embodiments, if there is an abnormal match between the bridge structure distribution information and the image format category, after obtaining second local influence degree information corresponding to each frame of the bridge monitoring inspection image from a second previously deployed abnormal matching record, the method further includes: calculating a number of categories of a second restrictive image format category corresponding to the second local influence magnitude information; and if the number of the types of the second restrictive image format type exceeds a second set type threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second restrictive image format type is an unmanned aerial vehicle filtering result.

It can be understood that, through the content, the unmanned aerial vehicle selection result of the inspection unmanned aerial vehicle to be selected can be determined according to the analysis basis of different layers, so that the accuracy and the reliability of the unmanned aerial vehicle selection result are ensured.

And S24, determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on the at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image.

In some possible embodiments, determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image includes: and if first target inspection influence information exceeding a first flight interference evaluation threshold exists in at least one first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image, determining that the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the first target inspection influence information is an unmanned aerial vehicle filtering result.

In other possible embodiments, determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on the at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image may include the following steps S241 to S243.

And S241, if second target inspection influence information which does not exceed a first flight interference evaluation threshold exists in at least one first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image, fusing the second target inspection influence information to obtain first overall inspection influence information.

And step S242, if the first integrity inspection influence degree information does not exceed a first global flight interference evaluation threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second target inspection influence degree information is an unmanned aerial vehicle allocation result.

And S243, if the first integrity inspection influence degree information exceeds a first global flight interference evaluation threshold value, selecting the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second target inspection influence degree information as an unmanned aerial vehicle filtering result.

It can be understood that unmanned aerial vehicle selection judgment is carried out through introducing integrity inspection influence degree information, unmanned aerial vehicle selection accuracy can be further improved, and selection omission or wrong selection is avoided.

And S25, determining the inspection unmanned aerial vehicle to be allocated according to the obtained unmanned aerial vehicle selection result.

In an actual implementation process, determining the inspection unmanned aerial vehicle to be deployed according to the obtained unmanned aerial vehicle selection result may include the following steps S251 and S252.

And S251, determining the unmanned aerial vehicle to be selected, the unmanned aerial vehicle selected result of which is the unmanned aerial vehicle allocation result, as the inspection unmanned aerial vehicle to be allocated.

And step S252, determining a bridge inspection flight path of an unmanned aerial vehicle cluster formed by the to-be-allocated inspection unmanned aerial vehicles according to the position distribution information of each to-be-allocated inspection unmanned aerial vehicle.

So, can ensure waiting of unmanned aerial vehicle cluster to allocate and patrol and examine unmanned aerial vehicle and patrol and examine the in-process and can not appear flying interference or conflict in the flight, can also realize carrying out the omnidirectional to appointed bridge monitoring area and patrol and examine.

In some possible embodiments, the determining, according to the location distribution information of each to-be-deployed inspection unmanned aerial vehicle, a bridge inspection flight path of a unmanned aerial vehicle cluster composed of the to-be-deployed inspection unmanned aerial vehicles in step S252 may include the following technical solutions described in steps S2521 to S2525.

Step S2521, acquiring the spatial three-dimensional state and flight attitude information of each to-be-allocated inspection unmanned aerial vehicle.

Step S2522, on the premise that the to-be-allocated inspection unmanned aerial vehicle contains the dynamic path tags according to the spatial three-dimensional state, determining correlation coefficients between the flight attitude information under the static path tags of the to-be-allocated inspection unmanned aerial vehicle and the flight attitude information under the dynamic path tags of the to-be-allocated inspection unmanned aerial vehicle according to the flight attitude information under the dynamic path tags of the plurality of to-be-allocated inspection unmanned aerial vehicles and the state description of the flight attitude information, and migrating the flight attitude information under the static path tags of the to-be-allocated inspection unmanned aerial vehicle, which is similar to the flight attitude information under the dynamic path tags, to the corresponding dynamic path tags.

Step S2523, on the premise that a plurality of flight attitude information are contained under the current static path tag of each to-be-allocated inspection unmanned aerial vehicle, determining a correlation coefficient among the flight attitude information under the current static path tag of each to-be-allocated inspection unmanned aerial vehicle according to the flight attitude information under the dynamic path tags of the plurality of to-be-allocated inspection unmanned aerial vehicles and the state description thereof, and grouping the flight attitude information under the current static path tag according to the correlation coefficient among the flight attitude information.

Step S2524, according to the flight attitude information and the state description thereof under the dynamic path labels of the plurality of to-be-deployed inspection unmanned aerial vehicles, setting a dynamic path label description for each group of flight attitude information obtained by grouping, and migrating each group of flight attitude information to the dynamic path label represented by the dynamic path label description.

Step S2525, determining the local flight path of each to-be-allocated inspection unmanned aerial vehicle according to the flight attitude information under the dynamic path labels, and performing global fusion on the local flight paths to obtain a bridge inspection flight path of an unmanned aerial vehicle cluster formed by the to-be-allocated inspection unmanned aerial vehicles.

Therefore, through the content, the flight attitude information under different path labels can be updated and adjusted, so that the accuracy and the real-time performance of the obtained local flight path of the inspection unmanned aerial vehicle to be allocated are ensured, and the bridge inspection flight path is ensured not to have local flight path conflict.

In some optional embodiments, determining, according to the flight attitude information under the dynamic path labels of the multiple to-be-deployed inspection unmanned aerial vehicles and the state description thereof, a correlation coefficient between each flight attitude information under the static path label of each to-be-deployed inspection unmanned aerial vehicle and each flight attitude information under the dynamic path label of each to-be-deployed inspection unmanned aerial vehicle, and migrating the flight attitude information under the static path label of each to-be-deployed inspection unmanned aerial vehicle, which is similar to the flight attitude information under the dynamic path label, to a corresponding dynamic path label includes: calculating cosine similarity between the description characteristics of the flight attitude information under the static path label of each to-be-allocated inspection unmanned aerial vehicle and the flight attitude information under the dynamic path label of each to-be-allocated inspection unmanned aerial vehicle; and respectively judging whether the cosine similarity reaches a first similarity threshold value, and transferring the flight attitude information under the static path label of which the cosine similarity reaches the first similarity threshold value to a corresponding dynamic path label.

In some optional embodiments, the determining, according to the flight attitude information and the state description thereof under the dynamic path tags of the plurality of to-be-deployed inspection unmanned aerial vehicles, the correlation coefficient between the flight attitude information under the current static path tag of each to-be-deployed inspection unmanned aerial vehicle, and grouping the flight attitude information under the current static path tag according to the correlation coefficient between the flight attitude information includes: calculating cosine similarity between description characteristics of flight attitude information of each to-be-allocated inspection unmanned aerial vehicle under the current static path label; and for one piece of flight attitude information under the current static path label of each inspection unmanned aerial vehicle to be allocated, dividing the flight attitude information and all pieces of flight attitude information of which the cosine similarity with the description characteristics reaches a second similarity threshold into a group.

In some possible embodiments, after collecting at least one frame of bridge monitoring inspection image of the inspection drone to be selected as described in step S21, the method further includes: judging whether the at least one frame of bridge monitoring inspection image contains significant content, wherein the significant content comprises: bridge structure key features or bridge detection equipment information; if each frame of bridge monitoring inspection image contains the significance content, second inspection influence information corresponding to each frame of bridge monitoring inspection image is obtained from a relational database which is deployed in advance so as to determine at least one second inspection influence information corresponding to at least one frame of bridge monitoring inspection image. On this basis, the determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on the at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image, which is described in step S24, includes: and determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to the at least one first inspection influence degree information and the at least one second inspection influence degree information. Therefore, the second patrol inspection influence degree information can be additionally introduced to carry out unmanned aerial vehicle selection analysis, and therefore the reliability of unmanned aerial vehicle selection analysis is ensured.

In some possible embodiments, after collecting at least one frame of bridge surveillance inspection image to be selected to inspect the drone as described in step S21, the method further includes: receiving at least one feedback image fed back by aiming at the at least one frame of bridge monitoring inspection image; the bridge linkage intelligent inspection cloud platform receives the at least one frame of bridge monitoring inspection image and then processes the image to obtain at least one feedback image; identifying the at least one feedback image to obtain unmanned aerial vehicle selected reference information corresponding to each feedback image; and if the unmanned aerial vehicle selected reference information indicates that the data to be subjected to safety monitoring inspection are not inquired, obtaining third inspection influence degree information corresponding to each frame of bridge monitoring inspection image from a pre-deployed error feedback list so as to determine at least one third inspection influence degree information corresponding to at least one frame of bridge monitoring inspection image. Based on this, the determining, by the step S24, the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on the at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image includes: determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to the at least one first inspection influence information and the at least one third inspection influence information; or determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to the at least one first inspection influence degree information, the at least one second inspection influence degree information and the at least one third inspection influence degree information.

In some optional embodiments, the method further comprises: identifying each frame of bridge monitoring inspection image in the at least one frame of bridge monitoring inspection image, and respectively obtaining uniquely bound bridge monitoring inspection variables; calculating the number of the bridge monitoring inspection variables corresponding to each frame of bridge monitoring inspection image; and determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image according to at least one of the at least one second inspection influence degree information and the at least one third inspection influence degree information, the at least one first inspection influence degree information and the number of the bridge monitoring inspection variables.

On the basis of the above, after determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image, the method further includes the following steps.

(1) And if the unmanned aerial vehicle selected result of the at least one frame of bridge monitoring inspection image is an unmanned aerial vehicle filtering result, deleting the at least one frame of bridge monitoring inspection image.

(2) And if the unmanned aerial vehicle selected result of the at least one frame of bridge monitoring inspection image is an unmanned aerial vehicle filtering result and at least one feedback image fed back by aiming at the at least one frame of bridge monitoring inspection image is received, deleting the at least one feedback image.

(3) And if the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image is an unmanned aerial vehicle allocation result and at least one feedback image fed back by aiming at the at least one frame of bridge monitoring inspection image is received, storing the at least one feedback image.

Therefore, the related patrol images can be deleted and stored, and follow-up patrol and allocation analysis basis and optimization basis are facilitated.

Based on the same inventive concept, please refer to fig. 3 in combination, a bridge linkage intelligent inspection system 30 is further provided, which comprises a bridge linkage intelligent inspection cloud platform 31 and an inspection unmanned aerial vehicle 32, which are communicated with each other. The bridge linkage intelligent inspection cloud platform 31 receives at least one frame of bridge monitoring inspection image of the unmanned aerial vehicle to be selected for inspection; identifying each frame of bridge monitoring inspection image in the at least one frame of bridge monitoring inspection image, and respectively obtaining at least two of corresponding bridge structure distribution information, image format types and bridge monitoring inspection angles; if at least two of the bridge monitoring inspection angle, the bridge structure distribution information and the image format category are in abnormal matching, obtaining first inspection influence information corresponding to each frame of bridge monitoring inspection image from an abnormal matching record which is deployed in advance; determining an unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image; and determining the inspection unmanned aerial vehicle to be allocated according to the obtained unmanned aerial vehicle selection result.

Based on the technical scheme, the bridge structure distribution information in the bridge monitoring inspection image, the abnormal matching records between the image format types and the bridge monitoring inspection angles can be extracted to serve as the unmanned aerial vehicle selection basis of multiple dimensions, whether the unmanned aerial vehicle to be selected is the unmanned aerial vehicle to be allocated can be determined, in the process of unmanned aerial vehicle inspection cluster linkage allocation analysis of the unmanned aerial vehicle to be allocated, the diversification and the richness of the linkage allocation analysis basis can be ensured, the cluster inspection linkage allocation accuracy and the credibility of the unmanned aerial vehicle to be allocated can be improved to a certain extent, and the unmanned aerial vehicle to be allocated can be used for carrying out comprehensive and comprehensive inspection on the designated bridge monitoring area.

In the several embodiments provided in the embodiments of the present application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus and method embodiments described above are illustrative only, as the flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

In addition, functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

The functions may be stored in a computer-readable storage medium if they are implemented in the form of software functional modules and sold or used as separate products. Based on such understanding, the technical solution of the present application, or portions thereof that substantially contribute to the prior art, may be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, the bridge-linked intelligent inspection cloud platform 10, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes. It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made to the present application by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. The utility model provides a bridge linkage intelligence inspection method based on unmanned aerial vehicle cluster which characterized in that, is applied to bridge linkage intelligence inspection cloud platform among the bridge linkage intelligence inspection system, bridge linkage intelligence inspection cloud platform with patrol inspection unmanned aerial vehicle communication connection among the bridge linkage intelligence inspection system, the method includes:

if at least two of the bridge monitoring inspection angle, the bridge structure distribution information and the image format category are in abnormal matching, acquiring first inspection influence information corresponding to each frame of bridge monitoring inspection image from an abnormal matching record which is deployed in advance;

determining an inspection unmanned aerial vehicle to be allocated according to the obtained unmanned aerial vehicle selection result;

the first inspection influence degree information corresponding to each frame of bridge monitoring inspection image comprises: first and second local influence degree information;

if abnormal matching exists between at least two of the bridge monitoring inspection angle, the bridge structure distribution information and the image format type, first inspection influence degree information corresponding to each frame of bridge monitoring inspection image is obtained from an abnormal matching record which is deployed in advance, and the method comprises the following steps:

if an abnormal match exists between the bridge monitoring inspection angle and the image format type, obtaining first local influence degree information corresponding to each frame of bridge monitoring inspection image from a first previously deployed abnormal matching record, wherein the obtaining comprises:

if there is an abnormal match between the bridge structure distribution information and the image format type, obtaining second local influence degree information corresponding to each frame of bridge monitoring inspection image from a second pre-deployed abnormal matching record, including:

wherein, if there is an abnormal match between the bridge monitoring inspection angle and the image format type, after obtaining the first local influence degree information corresponding to each frame of bridge monitoring inspection image from the first deployed abnormal matching record, the method further comprises:

if the number of the types of the second restrictive image format type exceeds a second set type threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second restrictive image format type is an unmanned aerial vehicle filtering result;

according to the selected result of unmanned aerial vehicle who obtains, confirm to wait to allocate and patrol and examine unmanned aerial vehicle, include:

and determining each local flight path of the unmanned aerial vehicle to be allocated and inspected according to the flight attitude information under the dynamic path label, and performing global fusion on the local flight paths to obtain a bridge inspection flight path of an unmanned aerial vehicle cluster formed by the unmanned aerial vehicles to be allocated and inspected.

2. The method of claim 1, wherein after collecting at least one frame of bridge surveillance inspection image to be selected for inspection of the drone, the method further comprises:

3. The method of claim 1 or 2, wherein after collecting at least one frame of bridge surveillance inspection image to be selected for inspecting the drone, the method further comprises:

receiving at least one feedback image fed back by aiming at the at least one frame of bridge monitoring inspection image; the bridge linkage intelligent inspection cloud platform receives the at least one frame of bridge monitoring inspection image and then processes the image to obtain at least one feedback image;

wherein the method further comprises:

4. The method according to claim 1, wherein the determining the unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on the at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image comprises:

and if first target patrol inspection influence information exceeding a first flight interference evaluation threshold exists in at least one piece of first patrol inspection influence information corresponding to the at least one frame of bridge monitoring patrol inspection image, determining that the unmanned aerial vehicle selection result of the bridge monitoring patrol inspection image corresponding to the first target patrol inspection influence information is an unmanned aerial vehicle filtering result.

5. The method according to claim 1, wherein the determining the unmanned aerial vehicle selection result of the at least one bridge monitoring inspection image based on the at least one first inspection influence information corresponding to the at least one bridge monitoring inspection image comprises:

if the first overall inspection influence degree information does not exceed a first overall flight interference evaluation threshold, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second target inspection influence degree information is an unmanned aerial vehicle allocation result;

and if the first integrity inspection influence information exceeds a first global flight interference evaluation threshold value, the unmanned aerial vehicle selection result of the bridge monitoring inspection image corresponding to the second target inspection influence information is an unmanned aerial vehicle filtering result.

6. The method of claim 1, wherein after determining the drone selection of the at least one frame of bridge surveillance inspection image, the method further comprises:

7. A bridge linkage intelligent inspection system is characterized by comprising a bridge linkage intelligent inspection cloud platform and an inspection unmanned aerial vehicle which are communicated with each other;

the inspection unmanned aerial vehicle to be selected is used for sending at least one frame of bridge monitoring inspection image to the bridge linkage intelligent inspection cloud platform;

the bridge linkage intelligent patrol inspection cloud platform is used for identifying each frame of bridge monitoring patrol inspection image in the at least one frame of bridge monitoring patrol inspection image and respectively obtaining at least two of corresponding bridge structure distribution information, image format types and bridge monitoring patrol inspection angles; if at least two of the bridge monitoring inspection angle, the bridge structure distribution information and the image format category are in abnormal matching, acquiring first inspection influence information corresponding to each frame of bridge monitoring inspection image from an abnormal matching record which is deployed in advance; determining an unmanned aerial vehicle selection result of the at least one frame of bridge monitoring inspection image based on at least one piece of first inspection influence information corresponding to the at least one frame of bridge monitoring inspection image; determining an inspection unmanned aerial vehicle to be allocated according to the obtained unmanned aerial vehicle selection result;

if the image format type and the bridge monitoring inspection angle have adaptive first target abnormal matching records in first deployed abnormal matching records, indicating to obtain first local influence degree information corresponding to the first target abnormal matching records from the previously deployed abnormal matching records;

wherein, if there is an abnormal match between the bridge structure distribution information and the image format type, after obtaining second local influence degree information corresponding to each frame of the bridge monitoring inspection image from a second pre-deployed abnormal matching record, the method further comprises:

wherein, according to every wait to allocate and patrol and examine unmanned aerial vehicle's position distribution information, confirm by wait to allocate and patrol and examine the bridge of the unmanned aerial vehicle cluster that unmanned aerial vehicle constitutes and patrol and examine flight path, include:

acquiring the space three-dimensional state and flight attitude information of each inspection unmanned aerial vehicle to be allocated;

8. The bridge linkage intelligent patrol cloud platform is characterized by comprising a processor, a network module and a memory; the processor and the memory communicate via the network module, and the processor reads the computer program from the memory and runs it to perform the method of any of the preceding claims 1-6.