CN115346138A - Target detection method, device and equipment of aerial image based on unmanned aerial vehicle - Google Patents
Target detection method, device and equipment of aerial image based on unmanned aerial vehicle Download PDFInfo
- Publication number
- CN115346138A CN115346138A CN202210842506.1A CN202210842506A CN115346138A CN 115346138 A CN115346138 A CN 115346138A CN 202210842506 A CN202210842506 A CN 202210842506A CN 115346138 A CN115346138 A CN 115346138A
- Authority
- CN
- China
- Prior art keywords
- image
- conversion
- aerial
- images
- target detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/17—Terrestrial scenes taken from planes or by drones
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Medical Informatics (AREA)
- Databases & Information Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- General Engineering & Computer Science (AREA)
- Molecular Biology (AREA)
- Remote Sensing (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of image detection, in particular to a target detection method of aerial images based on an unmanned aerial vehicle, which comprises the following steps: acquiring a plurality of aerial images of the unmanned aerial vehicle and mark data corresponding to each aerial image; performing image conversion processing on each aerial image according to a plurality of preset conversion type methods to obtain a plurality of conversion images corresponding to each aerial image and conversion type identifications corresponding to each conversion image; converting the position data corresponding to the target pixel according to the conversion type identifier of the conversion image to obtain mark data of the conversion image converted according to the corresponding conversion type; inputting a plurality of aerial images, the mark data corresponding to each aerial image, a plurality of conversion images obtained by performing image conversion on each aerial image according to a plurality of preset conversion type methods and the mark data corresponding to each conversion image into a neural network to be trained for training, and obtaining a target detection model.
Description
Technical Field
The invention relates to the technical field of image detection, in particular to a target detection method, a target detection device, target detection equipment and a storage medium for aerial images based on an unmanned aerial vehicle.
Background
At present, various problems can be frequently encountered in an aerial image of an unmanned aerial vehicle, such as haze, mountain shadow, over-dark image, over-exposed image, too-small target, complex background, large visual field, rotation and the like, and the aerial image is affected by factors such as shooting height, flight speed, weather, different reflection angles, unequal light receiving uniformity, electromagnetic interference and the like, so that the phenomena of color distortion, more noise points, overlooking of the image and the like occur in the aerial image, the quality of the aerial image is seriously affected, the target object is weakened or shielded, and important characteristic information of image data is difficult to obtain.
In addition, a large number of similar objects often exist in a target dense area in an aerial image of the unmanned aerial vehicle, so that missing detection or false alarm in detection is increased, and the accuracy and efficiency of target detection of the aerial image are reduced.
Disclosure of Invention
Based on this, an object of the present invention is to provide a target detection method, an apparatus, a device, and a storage medium for an aerial image based on an unmanned aerial vehicle, which are configured to obtain a conversion image obtained by performing image conversion processing on a plurality of conversion types corresponding to the aerial image of the unmanned aerial vehicle and a conversion type identifier corresponding to each conversion image, convert position data corresponding to a target pixel associated with a target detection object according to the conversion type identifier, obtain label data of the conversion image converted according to the corresponding conversion type, improve accuracy and efficiency of label data acquisition, and train a neural network model by using, as training data, label data corresponding to the aerial image, a plurality of conversion images obtained by performing image conversion processing on each aerial image according to a plurality of preset conversion types, and label data corresponding to each conversion image, so as to improve accuracy, robustness, and generalization capability of the aerial image, thereby improving accuracy and efficiency of target detection on the aerial image.
In a first aspect, an embodiment of the present application provides a target detection method for an aerial image based on an unmanned aerial vehicle, including the following steps:
acquiring a plurality of aerial images of an unmanned aerial vehicle and mark data corresponding to each aerial image, wherein the aerial images comprise preset images of target detection objects, and the mark data comprise a plurality of position data corresponding to target pixels associated with the target detection objects;
performing image conversion processing on each aerial image according to a plurality of preset conversion type methods to obtain a plurality of conversion images corresponding to each aerial image and a conversion type identifier corresponding to each conversion image;
converting the position data corresponding to the target pixel associated with the target detection object according to the conversion type identifier of the conversion image to obtain the mark data of the conversion image converted according to the corresponding conversion type;
inputting a plurality of aerial images, the mark data corresponding to each aerial image, a plurality of conversion images obtained by performing image conversion on each aerial image according to a plurality of preset conversion type methods and the mark data corresponding to each conversion image into a neural network to be trained for training to obtain a target detection model;
responding to a detection instruction, acquiring an aerial image to be detected of the unmanned aerial vehicle, inputting the aerial image to be detected into the target detection model, and acquiring a target detection result corresponding to the aerial image to be detected.
In a second aspect, an embodiment of the present application provides an object detection device based on an aerial image of an unmanned aerial vehicle, including:
the unmanned aerial vehicle monitoring system comprises an acquisition module, a monitoring module and a monitoring module, wherein the acquisition module is used for acquiring a plurality of aerial images of the unmanned aerial vehicle and mark data corresponding to each aerial image, the aerial images comprise preset images of target detection objects, and the mark data comprise a plurality of position data corresponding to target pixels associated with the target detection objects;
the image conversion module is used for respectively carrying out image conversion processing on each aerial image according to a plurality of preset conversion type methods to obtain a plurality of conversion images corresponding to each aerial image and conversion type identifications corresponding to each conversion image;
the marking data conversion module is used for converting the position data corresponding to the target pixel associated with the target detection object according to the conversion type identifier of the conversion image to obtain the marking data of the conversion image converted according to the corresponding conversion type;
the training module is used for inputting a plurality of aerial images, the mark data corresponding to each aerial image, a plurality of conversion images obtained by performing image conversion on each aerial image according to a plurality of preset conversion type methods and the mark data corresponding to each conversion image into a neural network to be trained for training to obtain a target detection model;
and the detection module is used for responding to a detection instruction, acquiring an aerial image to be detected of the unmanned aerial vehicle, inputting the aerial image to be detected into the target detection model, and acquiring a target detection result corresponding to the aerial image to be detected.
In a third aspect, an embodiment of the present application provides a computer device, including: a processor, a memory, and a computer program stored on the memory and executable on the processor; the computer program when executed by the processor realizes the steps of the method for object detection based on aerial images of drones according to the first aspect.
In a fourth aspect, the present application provides a storage medium storing a computer program, which when executed by a processor, implements the steps of the method for detecting an object based on an aerial image of a drone according to the first aspect.
In the embodiment of the application, a target detection method, a device, equipment and a storage medium of an aerial image based on an unmanned aerial vehicle are provided, a conversion image obtained by performing image conversion processing on a plurality of conversion type methods corresponding to the aerial image of the unmanned aerial vehicle and a conversion type identifier corresponding to each conversion image are obtained, position data corresponding to a target pixel associated with a target detection object are converted according to the conversion type identifier, mark data of the conversion image converted according to the corresponding conversion type are obtained, the accuracy and the efficiency of mark data obtaining are improved, mark data corresponding to the aerial image, a plurality of conversion images obtained by performing image conversion processing on the aerial image according to a plurality of preset conversion type methods and mark data corresponding to each conversion image are used as training data to train a neural network model, and the accuracy, the robustness and the generalization capability of the trained neural network model are improved, so that the accuracy and the efficiency of target detection on the aerial image are improved.
For a better understanding and practice, the invention is described in detail below with reference to the accompanying drawings.
Drawings
Fig. 1 is a schematic flowchart of a target detection method based on aerial images of an unmanned aerial vehicle according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a target detection method S1 in an aerial image based on an unmanned aerial vehicle according to an embodiment of the present application;
fig. 3 is a schematic flowchart of S3 in a target detection method based on an aerial image of an unmanned aerial vehicle according to an embodiment of the present application;
fig. 4 is a schematic flowchart of S4 in the target detection method based on an aerial image of an unmanned aerial vehicle according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a target detection device based on an aerial image of a drone according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if as used herein may be interpreted as" at "8230; \8230when" or "when 8230; \823030, when" or "in response to a determination", depending on the context.
Referring to fig. 1, fig. 1 is a schematic flow chart of a target detection method based on an aerial image of an unmanned aerial vehicle according to an embodiment of the present application, where the method includes the following steps:
s1: the method comprises the steps of obtaining a plurality of aerial images of the unmanned aerial vehicle and mark data corresponding to the aerial images.
The execution main body of the target detection method based on the aerial image of the unmanned aerial vehicle is detection equipment (hereinafter referred to as detection equipment for short) of the target detection method based on the aerial image of the unmanned aerial vehicle, and in an optional embodiment, the detection equipment can be one piece of computer equipment, a server or a server cluster formed by combining a plurality of pieces of computer equipment.
The aerial image comprises a preset image of a target detection object; in the embodiment, the detection equipment controls the unmanned aerial vehicle to carry out flight operation on a preset inspection route, and obtains video acquisition data on the inspection route;
in order to improve the operation efficiency, the detection device clips the video acquisition data, removes the video acquisition data which does not include the target detection object in the video acquisition data, obtains the clipped video acquisition data, performs frame extraction processing on the clipped video acquisition data, and obtains a plurality of images which include a preset target detection object as the aerial image, wherein the target detection object can be a person, a vehicle, a ship, a building and the like.
The tag data includes position data corresponding to a plurality of target pixels associated with the detected object, for example, the tag data may be an XML file, each detected object in the aerial image is framed with a rectangular information frame by setting a bndbox (feature information frame), pixels of the aerial image corresponding to the upper left corner and the lower right corner of the rectangular information frame are extracted, and position data corresponding to the pixels are obtained as tag data corresponding to the aerial image.
Referring to fig. 2, fig. 2 is a schematic flow diagram of S1 in a target detection method based on an aerial image of an unmanned aerial vehicle according to an embodiment of the present application, further including step S101, which is as follows:
s101: and carrying out interference judgment processing on the plurality of aerial images, and extracting a target aerial image from the plurality of aerial images.
In this embodiment, the detection device performs interference judgment processing on the plurality of aerial images, and extracts a target aerial image from the plurality of aerial images.
Specifically, the detection device performs binarization processing on the plurality of aerial images, extracts an object contour from the aerial images after the binarization processing, judges the number of identical pixels near one pixel in the aerial images after the binarization processing through the contour thickness to judge whether the pixel is an interference pixel, and if the number of the interference pixels in the aerial images is greater than or equal to a preset interference pixel number threshold value, judges that the aerial images are interference images, and discards the interference images; and if the number of the interference pixels of the aerial image is smaller than a preset interference pixel number threshold value, judging that the aerial image is a non-interference image and taking the non-interference image as the target aerial image for extraction.
S2: and respectively carrying out image conversion processing on each aerial image according to a plurality of preset conversion type methods to obtain a plurality of conversion images corresponding to each aerial image and conversion type identifications corresponding to each conversion image.
In this embodiment, the detection device performs image conversion processing on each of the plurality of aerial images according to a plurality of preset conversion type methods, to obtain a plurality of conversion images corresponding to each aerial image, and obtains a conversion type identifier corresponding to each conversion image according to the conversion type of the conversion image, where the image conversion processing includes scaling, rotation, tilting, flipping, gray scale adjustment, contrast adjustment, gaussian blur, and median blur processing.
Specifically, a zoomed image is obtained, such as by zooming the aerial image size; stretching the size of the aerial image to obtain a stretched image; obtaining an inclined image by adjusting the inclination angle; obtaining different gray level adjustment images by adjusting the gray level value of the image; turning the image horizontally and vertically to obtain turned images at different positions; obtaining contrast adjustment images under different contrasts by adjusting the contrast of the images; adjusting by a denoising and filtering method to eliminate noise points and noises of the image to obtain a smooth image; obtaining a Gaussian blur image through Gaussian blur; and obtaining a median blurred image through median blurring.
S3: and converting the position data corresponding to the target pixel associated with the target detection object according to the conversion type identifier of the conversion image to obtain the mark data of the conversion image converted according to the corresponding conversion type.
In this embodiment, the prediction device converts the position data corresponding to the target pixel associated with the target detection object according to the conversion type identifier of the conversion image, and obtains the label data of the conversion image converted according to the corresponding conversion type.
In an optional embodiment, the tag data is in a tree structure and includes a root node, the root node is provided with connected sub-nodes, each sub-node is provided with a branched sub-element, the sub-nodes store the identifiers of a plurality of detected objects in the aerial image, and the sub-elements store position data corresponding to target pixels associated with the detected objects.
Referring to fig. 3, fig. 3 is a schematic flow chart of S3 in the method for detecting a target based on an aerial image of an unmanned aerial vehicle according to an embodiment of the present application, including steps S301 to S302, which are as follows:
s301: the method comprises the steps of obtaining an identity label of a target detection object input by a user, retrieving from a root node of mark data of the aerial image according to the identity label of the target detection object, obtaining a target sub-node matched with the identity label of the target detection object, and obtaining position data corresponding to a target pixel associated with the target detection object from sub-elements of the target sub-node branch.
In order to improve the efficiency of acquiring the mark data of the target detection object, in this embodiment, the detection device acquires the identity of the target detection object input by the user, where the identity is a unique IP identifier and may be a number, a letter, or the like.
And retrieving from a root node of the marking data of the aerial image according to the identity of the target detection object, acquiring a target sub-node matched with the identity of the target detection object, and acquiring position data corresponding to a target pixel associated with the target detection object from sub-elements of the target sub-node branches.
S302: and according to the conversion type identifier of the conversion image, obtaining a proportional coefficient corresponding to the conversion type identifier, and according to the proportional coefficient corresponding to the conversion type identifier, converting position data corresponding to a target pixel associated with a target detection object in the mark data to obtain mark data corresponding to the conversion images of different conversion types.
In order to accurately acquire the tag data corresponding to the converted image and improve the efficiency of acquiring the tag data, in this embodiment, a scaling coefficient corresponding to a conversion type identifier is acquired according to the conversion type identifier of the converted image, and position data corresponding to a target pixel associated with a target detection object in the tag data is converted according to the scaling coefficient corresponding to the conversion type identifier, so as to acquire tag data corresponding to the converted image of each different conversion type. The workload of constructing the marking data corresponding to the conversion images of different conversion types is effectively reduced, the marking error caused by manual marking is avoided, and the efficiency and the accuracy of data expansion are improved.
S4: inputting a plurality of aerial images, the marking data corresponding to each aerial image, a plurality of conversion images obtained by performing image conversion on each aerial image according to a plurality of preset conversion type methods, and the marking data corresponding to each conversion image into a neural network to be trained for training, and obtaining a target detection model.
The neural network model to be trained adopts a YOLOv5 (You Only Look one) model, the YOLOv5 model is carried out based on an open source framework Pythrch, and the Pythroch framework integrates a plurality of neural network modules and call functions to define and form a detection algorithm.
By redefining the target detection as a classification, regression problem, inputting the entire image into a neural network module, dividing the image into grids, and predicting the class probability of each grid and generating a detection rectangular box.
In this embodiment, the detection device inputs a plurality of aerial images, the label data corresponding to each aerial image, a plurality of converted images obtained by performing image conversion processing on each aerial image according to a plurality of preset conversion type methods, and the label data corresponding to each converted image into a neural network to be trained for training, so as to obtain a target detection model, improve the robustness and generalization capability of the target detection model, and effectively reduce the occurrence of overfitting of the target detection model.
In an optional embodiment, the neural network model to be trained includes an image cropping module and a detection and recognition module which are connected in sequence. Referring to fig. 4, fig. 4 is a schematic flow chart of S4 in the method for detecting a target based on an aerial image of an unmanned aerial vehicle according to an embodiment of the present application, including steps S401 to S402, which are specifically as follows:
s401: and obtaining a feature cutting image corresponding to the aerial image output by the image cutting module and a feature cutting image corresponding to the conversion image according to the aerial image, the mark data corresponding to the aerial image, the conversion image, the mark data corresponding to the conversion image and the image cutting module.
In this embodiment, the detection device inputs the mark data corresponding to the aerial image and the mark data corresponding to the conversion image and the conversion image into the image cropping module, and obtains the feature cropping image corresponding to the aerial image and the feature cropping image corresponding to the conversion image output by the image cropping module, thereby effectively reducing redundancy of data input, improving the efficiency of model training, and compared with the conversion image, the feature information in the feature cropping image has higher proportion, and the feature cropping image is adopted as training data, so that the model can better learn the image features, and the accuracy of the target detection model is improved.
S402: and inputting the feature clipping image corresponding to the aerial image and the feature clipping image corresponding to the converted image into a detection and recognition module for iterative training to obtain the target detection model.
In this embodiment, the detection device inputs the feature clipping image corresponding to the aerial image and the feature clipping image corresponding to the converted image into the detection and recognition module for iterative training, obtains a plurality of trained neural network models, and obtains a target neural network model from the plurality of trained neural network models as the target detection model. Specifically, the detection device extracts feature vectors of features in the feature clipping image by using a detection recognition module in a neural network model, captures multi-scale context information and object boundary information through relevant information such as edges, colors and shapes of a target detection object, and performs iterative training to obtain the target detection model.
S4: responding to a detection instruction, acquiring an aerial image to be detected of the unmanned aerial vehicle, inputting the aerial image to be detected into the target detection model, and acquiring a target detection result corresponding to the aerial image to be detected.
The detection instruction is sent by a user and received by the detection equipment.
In this embodiment, the detection device acquires a detection instruction sent by a user, acquires an aerial image to be detected of the unmanned aerial vehicle, inputs the aerial image to be detected into the target detection model, and acquires a target detection result corresponding to the aerial image to be detected.
In an optional embodiment, the detection device obtains a target detection identifier corresponding to the to-be-detected aerial image according to the target detection result, returns the target detection identifier to a display interface of the detection device, and displays and marks the identification identifier on the to-be-detected aerial image.
Referring to fig. 5, fig. 5 is a schematic structural diagram of a target detection apparatus based on aerial images of a drone according to an embodiment of the present application, where the apparatus may implement all or a part of the target detection apparatus based on aerial images of a drone through software, hardware, or a combination of the two, and the apparatus 5 includes:
the unmanned aerial vehicle positioning system comprises an acquisition module 51, a positioning module and a control module, wherein the acquisition module 51 is used for acquiring a plurality of aerial images of the unmanned aerial vehicle and mark data corresponding to each aerial image, the aerial images comprise preset images of target detection objects, and the mark data comprise a plurality of position data corresponding to target pixels associated with the target detection objects;
an image conversion module 52, configured to perform image conversion processing on each of the aerial images according to a plurality of preset conversion type methods, so as to obtain a plurality of conversion images corresponding to each of the aerial images and a conversion type identifier corresponding to each of the conversion images
A tag data conversion module 53, configured to convert, according to the conversion type identifier of the conversion image, position data corresponding to a target pixel associated with the target detection object, and obtain tag data of the conversion image converted according to the corresponding conversion type;
a training module 54, configured to input a plurality of conversion images obtained by performing image conversion processing on a plurality of aerial images, the label data corresponding to each aerial image, and each aerial image according to a plurality of preset conversion type methods, and the label data corresponding to each conversion image into a neural network to be trained for training, so as to obtain a target detection model;
the detection module 55 is configured to, in response to a detection instruction, acquire an aerial image to be detected of the unmanned aerial vehicle, input the aerial image to be detected into the target detection model, and acquire a target detection result corresponding to the aerial image to be detected.
In the embodiment, a plurality of aerial images of the unmanned aerial vehicle and mark data corresponding to each aerial image are obtained through an obtaining module, wherein the aerial images comprise preset images of target detection objects, and the mark data comprise a plurality of position data corresponding to target pixels associated with the target detection objects; performing image conversion processing on each aerial image according to a plurality of preset conversion type methods through an image conversion module to obtain a plurality of conversion images corresponding to each aerial image and a conversion type identifier corresponding to each conversion image; converting position data corresponding to a target pixel associated with the target detection object according to the conversion type identifier of the conversion image through a mark data conversion module to obtain mark data of the conversion image converted according to the corresponding conversion type; inputting a plurality of aerial images, the mark data corresponding to each aerial image, a plurality of conversion images obtained by performing image conversion on each aerial image according to a plurality of preset conversion type methods and the mark data corresponding to each conversion image into a neural network to be trained for training through a training module to obtain a target detection model; the detection module responds to a detection instruction, acquires an aerial image to be detected of the unmanned aerial vehicle, inputs the aerial image to be detected into the target detection model, and acquires a target detection result corresponding to the aerial image to be detected.
The method comprises the steps of obtaining a conversion image obtained after image conversion processing is carried out by a plurality of conversion type methods corresponding to aerial images of a plurality of unmanned aerial vehicles and a conversion type identification corresponding to each conversion image, converting position data corresponding to target pixels associated with target detection objects according to the conversion type identification, obtaining mark data of the conversion image converted according to the corresponding conversion type, improving the accuracy and efficiency of mark data obtaining, training a neural network model by taking the mark data corresponding to the aerial images, a plurality of conversion images obtained after image conversion processing is carried out on the aerial images according to a plurality of preset conversion type methods and mark data corresponding to the conversion images as training data, and improving the accuracy, robustness and generalization capability of the trained neural network model, so that the accuracy and efficiency of target detection of the aerial images are improved.
Referring to fig. 6, fig. 6 is a schematic structural diagram of a computer device according to an embodiment of the present application, where the computer device 6 includes: a processor 61, a memory 62 and a computer program 63 stored on the memory 62 and executable on the processor 61; the computer device may store a plurality of instructions, where the instructions are suitable for being loaded by the processor 61 and executing the method steps in the embodiments shown in fig. 1 to fig. 4, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 to fig. 4, which is not described herein again.
The Memory 62 may include a Random Access Memory (RAM) 62, and may also include a Read-Only Memory (Read-Only Memory) 62. Optionally, the memory 62 includes a non-transitory computer-readable medium. The memory 62 may be used to store instructions, programs, code sets or instruction sets. The memory 62 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the various method embodiments described above, and the like; the storage data area may store data and the like referred to in the above respective method embodiments. The memory 62 may alternatively be at least one memory device located remotely from the aforementioned processor 61.
An embodiment of the present application further provides a storage medium, where the storage medium may store a plurality of instructions, where the instructions are suitable for being loaded by a processor and executing the method steps in the embodiments shown in fig. 1 to 4, and a specific execution process may refer to specific descriptions of the embodiments shown in fig. 1 to 4, which is not described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated module/unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc.
The present invention is not limited to the above-described embodiments, and various modifications and variations of the present invention are intended to be included within the scope of the claims and the equivalent technology of the present invention if they do not depart from the spirit and scope of the present invention.
Claims (10)
1. A target detection method of aerial images based on an unmanned aerial vehicle is characterized by comprising the following steps:
acquiring a plurality of aerial images of an unmanned aerial vehicle and mark data corresponding to each aerial image, wherein the aerial images comprise images of preset target detection objects, and the mark data comprise a plurality of position data corresponding to target pixels associated with the target detection objects;
performing image conversion processing on each aerial image according to a plurality of preset conversion type methods to obtain a plurality of conversion images corresponding to each aerial image and conversion type identifications corresponding to each conversion image;
converting position data corresponding to a target pixel associated with the target detection object according to the conversion type identifier of the conversion image to obtain mark data of the conversion image converted according to the corresponding conversion type;
inputting a plurality of aerial images, the mark data corresponding to each aerial image, a plurality of conversion images obtained by performing image conversion on each aerial image according to a plurality of preset conversion type methods and the mark data corresponding to each conversion image into a neural network to be trained for training to obtain a target detection model;
responding to a detection instruction, acquiring an aerial image to be detected of the unmanned aerial vehicle, inputting the aerial image to be detected into the target detection model, and acquiring a target detection result corresponding to the aerial image to be detected.
2. The method of claim 1, wherein the method comprises: the marking data are of a tree structure and comprise root nodes, the root nodes are provided with connected sub-nodes, each sub-node is provided with a branched sub-element, the sub-nodes store identity marks of a plurality of detection objects in the aerial image, and the sub-elements store position data corresponding to target pixels associated with the detection objects.
3. The target detection method based on the aerial image of the unmanned aerial vehicle of claim 2, wherein the step of converting the position data corresponding to the target pixel associated with the target detection object according to the conversion type identifier of the conversion image to obtain the mark data of the conversion image converted according to the corresponding conversion type comprises the steps of:
acquiring an identity of a target detection object input by a user, retrieving from a root node of marking data of the aerial image according to the identity of the target detection object, acquiring a target sub-node matched with the identity of the target detection object, and acquiring position data corresponding to a target pixel associated with the target detection object from sub-elements of target sub-node branches;
and according to the conversion type identifier of the conversion image, acquiring a proportionality coefficient corresponding to the conversion type identifier, wherein the proportionality coefficient is a parameter in the image conversion processing step, and according to the proportionality coefficient corresponding to the conversion type identifier, converting position data corresponding to a target pixel associated with a target detection object in the mark data to acquire mark data corresponding to the conversion images of different conversion types.
4. The method for detecting the target of the aerial image based on the unmanned aerial vehicle as claimed in claim 1, wherein: the neural network model to be trained comprises an image cutting module and a detection and identification module which are sequentially connected.
5. The target detection method of aerial images based on unmanned aerial vehicles according to claim 4, wherein the method comprises the steps of inputting a plurality of aerial images, the label data corresponding to each aerial image, a plurality of converted images obtained by subjecting each aerial image to image conversion processing according to a plurality of preset conversion types, and the label data corresponding to each converted image into a neural network to be trained for training, and obtaining a target detection model, and comprises the following steps:
according to the aerial image, the mark data corresponding to the aerial image, the conversion image, the mark data corresponding to the conversion image and the image cutting module, obtaining a feature cutting image corresponding to the aerial image and a feature cutting image corresponding to the conversion image which are output by the image cutting module;
and inputting the feature clipping image corresponding to the aerial image and the feature clipping image corresponding to the converted image into a detection and recognition module for iterative training to obtain the target detection model.
6. The method of claim 1, wherein the step of obtaining a plurality of aerial images of the drone further comprises the steps of:
and performing interference judgment processing on the plurality of aerial images, and extracting a target aerial image from the plurality of aerial images.
7. The method of claim 1, wherein the method comprises: the conversion type method comprises zooming, rotating, inclining, turning, gray scale adjusting, contrast adjusting, noise filtering, gaussian blurring and median blurring; the conversion image comprises a zoom image, a rotation image, a tilt image, a flip image, a gray-scale adjustment image, a contrast adjustment image, a smooth image, a Gaussian blur image and a median blur image.
8. The utility model provides a target detection device based on unmanned aerial vehicle's image of taking photo by plane, its characterized in that includes:
the unmanned aerial vehicle aerial image acquisition module is used for acquiring a plurality of aerial images of the unmanned aerial vehicle and mark data corresponding to each aerial image, wherein the aerial images comprise preset images of target detection objects, and the mark data comprise a plurality of position data corresponding to target pixels associated with the target detection objects;
an image conversion module, configured to perform image conversion processing on each of the aerial images according to a plurality of preset conversion type methods, to obtain a plurality of conversion images corresponding to each of the aerial images and a conversion type identifier corresponding to each of the conversion images
A label data conversion module, configured to convert, according to the conversion type identifier of the conversion image, position data corresponding to a target pixel associated with the target detection object, and obtain label data of the conversion image converted according to the corresponding conversion type;
the training module is used for inputting a plurality of aerial images, the mark data corresponding to the aerial images, a plurality of conversion images obtained by performing image conversion on the aerial images according to a plurality of preset conversion type methods and the mark data corresponding to the conversion images into a neural network to be trained for training to obtain a target detection model;
and the detection module is used for responding to a detection instruction, acquiring an aerial image to be detected of the unmanned aerial vehicle, inputting the aerial image to be detected into the target detection model, and acquiring a target detection result corresponding to the aerial image to be detected.
9. A computer device, comprising: a processor, a memory, and a computer program stored on the memory and executable on the processor; the computer program when executed by the processor implements the steps of the method of object detection based on aerial images of drones according to any one of claims 1 to 7.
10. A storage medium, characterized by: the storage medium stores a computer program which, when executed by a processor, implements the steps of the method for object detection based on aerial images of drones of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210842506.1A CN115346138A (en) | 2022-07-18 | 2022-07-18 | Target detection method, device and equipment of aerial image based on unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210842506.1A CN115346138A (en) | 2022-07-18 | 2022-07-18 | Target detection method, device and equipment of aerial image based on unmanned aerial vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115346138A true CN115346138A (en) | 2022-11-15 |
Family
ID=83949743
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210842506.1A Pending CN115346138A (en) | 2022-07-18 | 2022-07-18 | Target detection method, device and equipment of aerial image based on unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115346138A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116721095A (en) * | 2023-08-04 | 2023-09-08 | 杭州瑞琦信息技术有限公司 | Aerial photographing road illumination fault detection method and device |
-
2022
- 2022-07-18 CN CN202210842506.1A patent/CN115346138A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116721095A (en) * | 2023-08-04 | 2023-09-08 | 杭州瑞琦信息技术有限公司 | Aerial photographing road illumination fault detection method and device |
| CN116721095B (en) * | 2023-08-04 | 2023-11-03 | 杭州瑞琦信息技术有限公司 | Aerial photographing road illumination fault detection method and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110060237B (en) | Fault detection method, device, equipment and system | |
| CN111178183B (en) | Face detection method and related device | |
| CN110287963B (en) | OCR recognition method for comprehensive performance test | |
| CN108564579B (en) | Concrete crack detection method and detection device based on time-space correlation | |
| CN112669344A (en) | Method and device for positioning moving object, electronic equipment and storage medium | |
| CN111860439A (en) | Unmanned aerial vehicle inspection image defect detection method, system and equipment | |
| CN107704797B (en) | Real-time detection method, system and equipment based on pedestrians and vehicles in security video | |
| CN111598889B (en) | Identification method and device for inclination fault of equalizing ring and computer equipment | |
| CN111339902B (en) | Liquid crystal display indication recognition method and device for digital display instrument | |
| CN111680690A (en) | Character recognition method and device | |
| CN110287787B (en) | Image recognition method, image recognition device and computer-readable storage medium | |
| CN113435407B (en) | Small target identification method and device for power transmission system | |
| CN112686172A (en) | Method and device for detecting foreign matters on airport runway and storage medium | |
| CN112330597A (en) | Image difference detection method and device and computer equipment | |
| CN112784835A (en) | Method and device for identifying authenticity of circular seal, electronic equipment and storage medium | |
| CN115346138A (en) | Target detection method, device and equipment of aerial image based on unmanned aerial vehicle | |
| CN115311623A (en) | Equipment oil leakage detection method and system based on infrared thermal imaging | |
| CN111461076A (en) | Smoke detection method and smoke detection system combining frame difference method and neural network | |
| CN114973057A (en) | Video image detection method based on artificial intelligence and related equipment | |
| CN113537017A (en) | Optical remote sensing image airplane detection method and device based on cascade regression correction | |
| CN111582012A (en) | Method and device for detecting small target ship | |
| CN111079752A (en) | Method and device for identifying circuit breaker in infrared image and readable storage medium | |
| CN116152191A (en) | Display screen crack defect detection method, device and equipment based on deep learning | |
| CN115345895A (en) | Image segmentation method and device for visual detection, computer equipment and medium | |
| CN114299020A (en) | Scooter display screen defect detection method and system based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |