CN116128883A - Photovoltaic panel quantity counting method and device, electronic equipment and storage medium - Google Patents

Photovoltaic panel quantity counting method and device, electronic equipment and storage medium Download PDF

Info

Publication number
CN116128883A
CN116128883A CN202310420646.4A CN202310420646A CN116128883A CN 116128883 A CN116128883 A CN 116128883A CN 202310420646 A CN202310420646 A CN 202310420646A CN 116128883 A CN116128883 A CN 116128883A
Authority
CN
China
Prior art keywords
target object
frame
image
target
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310420646.4A
Other languages
Chinese (zh)
Inventor
洪流
柴东元
温招洋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Snegrid Electric Technology Co ltd
Original Assignee
Snegrid Electric Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Snegrid Electric Technology Co ltd filed Critical Snegrid Electric Technology Co ltd
Priority to CN202310420646.4A priority Critical patent/CN116128883A/en
Publication of CN116128883A publication Critical patent/CN116128883A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y04—INFORMATION OR COMMUNICATION TECHNOLOGIES HAVING AN IMPACT ON OTHER TECHNOLOGY AREAS
    • Y04S—SYSTEMS INTEGRATING TECHNOLOGIES RELATED TO POWER NETWORK OPERATION, COMMUNICATION OR INFORMATION TECHNOLOGIES FOR IMPROVING THE ELECTRICAL POWER GENERATION, TRANSMISSION, DISTRIBUTION, MANAGEMENT OR USAGE, i.e. SMART GRIDS
    • Y04S10/00—Systems supporting electrical power generation, transmission or distribution
    • Y04S10/50—Systems or methods supporting the power network operation or management, involving a certain degree of interaction with the load-side end user applications

Abstract

The invention discloses a photovoltaic panel quantity counting method, which comprises the following steps: acquiring a video of an area to be counted; selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set; inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image; the number of target objects in each frame of image of the video is counted based on the target detector. The construction supervision unit only needs to provide a construction site video, the result can be automatically counted by the photovoltaic panel quantity counting method based on target tracking, the operation is simple and convenient, compared with a manual counting method, the efficiency is improved by more than 3 times, and the working intensity of staff is greatly reduced.

Description

Photovoltaic panel quantity counting method and device, electronic equipment and storage medium
Technical Field
The patent application relates to the technical field of artificial intelligence, in particular to a photovoltaic panel quantity counting method, a device, electronic equipment and a storage medium.
Background
At present, the construction progress statistics of the photovoltaic station mainly utilizes unmanned aerial vehicle shooting field video manual statistics, construction supervision personnel utilize unmanned aerial vehicle high altitude to beat photovoltaic construction field, identify photovoltaic panels through human eyes, and count out photovoltaic panel quantity.
When the number of the photovoltaic panels is counted manually, the human eyes need to watch videos carefully from beginning to end, the manual counting time length is about 3 times or more of the video time length, and when the human eyes do the work for a long time, the human eyes are easy to fatigue so as to make mistakes, so that the manual counting work efficiency is low and the accuracy is low.
Disclosure of Invention
The invention aims to solve one of the technical problems in the related art to at least a certain extent, and therefore, the first aim of the invention is to provide a photovoltaic panel quantity counting method, and only a construction site video is needed to be provided, and the photovoltaic panel quantity counting method based on target tracking can automatically count the result, so that the operation is simple and convenient, the efficiency is improved by more than 3 times compared with the manual counting method, and the working intensity of staff is greatly reduced.
A second object of the present invention is to provide a photovoltaic panel counting device.
A third object of the present invention is to propose an electronic device.
A fourth object of the present invention is to propose a storage medium.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a method for counting the number of photovoltaic panels, including the following steps:
acquiring a video of an area to be counted;
selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
the number of target objects in each frame of image of the video is counted based on the target detector.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, partial images are selected based on each frame of video image, a target object in the selected images is marked with an external rectangular frame, and a training data set is manufactured, and the method comprises the following steps:
selecting partial images based on each frame of images of the video, labeling an external rectangular frame on a target object in the images, and manufacturing a training data set, wherein the external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, and the rectangular frame information is stored.
According to the method for counting the number of the photovoltaic panels, the training data set of the circumscribed rectangular frame is input into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image, and the method comprises the following steps:
the training data set of the external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module carries out feature fusion on the first feature map and outputs a second feature map, the Head module carries out convolution on the second feature map, the offset of a target object prediction frame is output, and the target detector of the target object is generated by adopting NMS non-maximum suppression.
According to the method for counting the number of the photovoltaic panels, the counting of the number of the target objects of each frame of image of the video based on the target detector comprises the following steps:
inputting each frame of video image of the region to be counted into a target detector, and obtaining the apparent characteristics of a target object in each frame of image;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
Matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
and counting the number set of each target object qualified in matching, and calculating the number of the target objects.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, each frame of image of the video of the area to be counted is input into the target detector, and the apparent characteristics of the target object in each frame of image are obtained, and the method comprises the following steps:
inputting the video of the region to be counted into a target detector, wherein the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target object
And category information collection
Coordinate information set
The expression is as follows: the invention aims to solve one of the technical problems in the related art to at least a certain extent, and therefore, the first aim of the invention is to provide a photovoltaic panel quantity counting method, and only a construction site video is needed to be provided, and the photovoltaic panel quantity counting method based on target tracking can automatically count the result, so that the operation is simple and convenient, the efficiency is improved by more than 3 times compared with the manual counting method, and the working intensity of staff is greatly reduced.
A second object of the present invention is to provide a photovoltaic panel counting device.
A third object of the present invention is to propose an electronic device.
A fourth object of the present invention is to propose a storage medium.
In order to achieve the above object, an embodiment of a first aspect of the present invention provides a method for counting the number of photovoltaic panels, including the following steps:
acquiring a video of an area to be counted;
selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
the number of target objects in each frame of image of the video is counted based on the target detector.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, partial images are selected based on each frame of video image, a target object in the selected images is marked with an external rectangular frame, and a training data set is manufactured, and the method comprises the following steps:
selecting partial images based on each frame of images of the video, labeling an external rectangular frame on a target object in the images, and manufacturing a training data set, wherein the external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, and the rectangular frame information is stored.
According to the method for counting the number of the photovoltaic panels, the training data set of the circumscribed rectangular frame is input into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image, and the method comprises the following steps:
the training data set of the external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module carries out feature fusion on the first feature map and outputs a second feature map, the Head module carries out convolution on the second feature map, the offset of a target object prediction frame is output, and the target detector of the target object is generated by adopting NMS non-maximum suppression.
According to the method for counting the number of the photovoltaic panels, the counting of the number of the target objects of each frame of image of the video based on the target detector comprises the following steps:
inputting each frame of video image of the region to be counted into a target detector, and obtaining the apparent characteristics of a target object in each frame of image;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
Matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
and counting the number set of each target object qualified in matching, and calculating the number of the target objects.
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, each frame of image of the video of the area to be counted is input into the target detector, and the apparent characteristics of the target object in each frame of image are obtained, and the method comprises the following steps:
inputting the video of the region to be counted into a target detector, wherein the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target object
And category information collection
Coordinate information set
The expression is as follows:
Figure SMS_1
wherein ,
Figure SMS_2
is the t-th frame image I t A set of coordinate information of the detected target object;
Figure SMS_3
wherein ,
is the coordinate information of the i-th target object,
is the x coordinate value of the upper left corner of the circumscribed rectangle of the target object,
is the y coordinate value of the left upper corner of the circumscribed rectangle frame of the target object,
is the x coordinate value of the right lower corner of the circumscribed rectangle frame of the target object,
the y coordinate value of the right lower corner of the circumscribed rectangle frame of the target object; category information collection
Figure SMS_4
The expression is as follows:
Figure SMS_5
；
wherein ,
Figure SMS_6
is the category of the i-th target object, +. >
Figure SMS_7
The value 0 or 1,0 represents a complete target object, and 1 represents an incomplete target object; />
Creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker, comprising:
video first frame image I based on object detector 1 Detecting based on the detected coordinate information set of the target object
Figure SMS_8
Initializing and creating a Kalman tracker; kalman tracker generates a numbered set of target object coordinate information +.>
Figure SMS_9
，/>
Figure SMS_10
，/>
Figure SMS_11
Is the unique number of the ith target object, and predicts the second frame image I of the video 2 Number set of coordinate information->
Figure SMS_12
，
Figure SMS_13
，/>
Figure SMS_14
Is the predicted coordinates of the ith target object;
the calculation formula of the Kalman tracker is as follows:
Figure SMS_15
;
Figure SMS_16
;
Figure SMS_17
for the state information of the target object at time k-1, i.e.>
Figure SMS_18
, wherein ,/>
Figure SMS_19
) Is the center coordinates of the rectangular frame of the target object, s is the rectangular frame area, r is the rectangular frame aspect ratio,/->
Figure SMS_20
Is the abscissa speed, +.>
Figure SMS_21
Is the ordinate speed, +.>
Figure SMS_22
Is the area velocity; />
Figure SMS_23
Estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
each frame of image of the video of the area to be counted is operated according to the steps;
Matching the detection result of the apparent characteristic of any frame target object with the prediction result, comprising:
using KM algorithm, taking IOU distance as weight, and detecting the t frame
And prediction result
Matching, when IOU is larger than or equal to 0.1, utilizing the coordinate information of the target object matched by the t-th frame
Updating
While continuing to predict the t+1st frame target object coordinates
The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, unmatched target objects are utilized
Creating a new Kalman tracker and the number of the target object;
Figure SMS_24
estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
each frame of image of the video of the area to be counted is operated according to the steps;
matching the detection result of the apparent characteristic of any frame target object with the prediction result, comprising:
using KM algorithm, taking IOU distance as weight, and detecting the t frame
Figure SMS_25
And prediction result
Figure SMS_26
Matching, when IOU is larger than or equal to 0.1, utilizing the coordinate information of the target object matched by the t-th frame
Figure SMS_27
Update->
Figure SMS_28
While continuing to predict the t+1st frame target object coordinates +.>
Figure SMS_29
The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, using +.o. of unmatched target object>
Figure SMS_30
Creating a new Kalman tracker and the number of the target object;
Counting the number set of each target object qualified by matching, and calculating the number of the target objects, wherein the number set comprises the following steps:
sequentially counting target objects of each frame of image
Figure SMS_31
Numbering sets of individual ID occurrences
Figure SMS_32
and />
Figure SMS_33
Number set of occurrence frequency of each ID of 1 +.>
Figure SMS_34
, wherein ,
Figure SMS_35
the expression of (2) is as follows:
Figure SMS_36
，/>
Figure SMS_37
is numbered->
Figure SMS_38
The frequency of occurrence of the complete target object of (1) is the initial value, and 1 is accumulated when the complete target object occurs 1 time in one frame of image;
Figure SMS_39
the expression is as follows:
Figure SMS_40
，/>
Figure SMS_41
is numbered->
Figure SMS_42
The frequency of occurrence of incomplete target objects of (1) is the initial value, and 1 is accumulated when the incomplete target objects occur 1 time in one frame of image;
and counting the number of ID categories of which the frequency of occurrence of the target object exceeds 3 times based on the number set, obtaining the number of the target objects, and accumulating and counting to obtain the number of the target objects.
According to the photovoltaic panel quantity counting method provided by the embodiment of the invention, for the unmatched target object coordinate information
Figure SMS_43
Initializing and creating a new Kalman tracker, generating new numbers at the same time, and updating the number set of occurrence frequency +.>
Figure SMS_44
and />
Figure SMS_45
。
According to the method for counting the number of the photovoltaic panels, which is provided by the embodiment of the invention, the number of the target objects is obtained by counting the ID category number of which the frequency of occurrence of the target objects exceeds 3 times based on the number set, and the number of the target objects is obtained by accumulation and statistics, and the method comprises the following steps:
Based on numbered sets
Figure SMS_46
Statistics of->
Figure SMS_47
ID category number more than 3, obtaining the number Clsw of the complete target object;
based on numbered sets
Figure SMS_48
Statistics of->
Figure SMS_49
ID category number more than 3, obtaining the number Clsh of incomplete target objects;
the number of target objects M, m=clsw+clsh is counted.
An embodiment of a second aspect of the present invention provides a photovoltaic panel count apparatus, including:
the acquisition module is used for acquiring videos of the areas to be counted;
the training set making module is used for selecting partial images based on each frame of image of the video, labeling a target object in the selected images with an external rectangular frame, and making a training data set;
the training module is used for inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training so as to obtain a target detector capable of describing the apparent characteristics of a target object in each frame of image;
and the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
An embodiment of the third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, where the processor implements a method for counting the number of photovoltaic panels according to the embodiment of the first aspect of the present invention when the processor executes the program.
An embodiment of a fourth aspect of the present invention proposes a computer readable storage medium, on which a computer program is stored, which program, when being executed by a processor, implements a method for counting the number of photovoltaic panels as proposed by an embodiment of the first aspect of the present invention.
Compared with the prior art, the invention has the beneficial effects that: according to the photovoltaic panel number counting method, the target detector is obtained through training, the apparent characteristics of the target object in each frame of video image are obtained through the target detector, the detection result and the prediction result are matched, the ID category number is counted, and the number of the photovoltaic panels is counted. The construction supervision unit only needs to provide a construction site video, the result can be automatically counted by the photovoltaic panel quantity counting method based on target tracking, the operation is simple and convenient, compared with a manual counting method, the efficiency is improved by more than 3 times, and the working intensity of staff is greatly reduced.
Drawings
FIG. 1 is a flow chart of a method for counting the number of photovoltaic panels according to one embodiment of the present application;
FIG. 2 is a detailed flow chart of step S4 according to one embodiment of the present application;
FIG. 3 is a schematic view of a frame of an image captured in an embodiment of the present application;
FIG. 4 is a schematic view of a circumscribed rectangular frame of an image of a photovoltaic panel according to one embodiment of the present application;
FIG. 5 is a schematic diagram of target detector identification in one embodiment of the present application;
FIG. 6 is a schematic diagram of a Mosaic image enhancement technique in one embodiment of the present application;
FIG. 7 is a statistical flow chart of one embodiment of the present application;
fig. 8 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
Other advantages and effects of the present application will be readily apparent to those skilled in the art from the present disclosure, by describing embodiments of the present application with specific examples. This application is also intended to cover any adaptations or uses of various embodiments and can be practiced in different but specific details of the subject matter within the scope of the description and from various points of view. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
Other advantages and effects of the present application will be readily apparent to those skilled in the art from the present disclosure, by describing embodiments of the present application with specific examples. This application is also intended to cover any adaptations or uses of various embodiments and can be practiced in different but specific details of the subject matter within the scope of the description and from various points of view. It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
The following describes in detail a method, an apparatus, an electronic device, and a storage medium for counting the number of photovoltaic panels provided in the embodiments of the present application through specific embodiments and application scenarios thereof with reference to the accompanying drawings.
In the embodiment of the application, the number of the photovoltaic panels can be calculated by constructing a photovoltaic panel number counting device, executing a photovoltaic panel number counting method in the photovoltaic panel number counting device, obtaining a target detector through training, obtaining apparent characteristics of a target object in each frame of image of a video through the target detector, matching a detection result and a prediction result, counting the number of ID categories, and calculating the number of the counted photovoltaic panels. The result can be automatically counted by only providing the construction site video and the photovoltaic panel quantity counting method based on target tracking, so that the operation is simple and convenient, the efficiency is improved by more than 3 times compared with the manual counting method, and the working intensity of staff is greatly reduced.
The photovoltaic panel quantity counting device can comprise an acquisition module, a training set making module, a training module and a counting module.
The acquisition module is used for acquiring videos of the areas to be counted;
the training set making module is used for selecting partial images based on each frame of image of the video, labeling a target object in the selected images with an external rectangular frame, and making a training data set;
The training module is used for inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training so as to obtain a target detector capable of describing the apparent characteristics of a target object in each frame of image;
and the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
The device can be applied to a terminal, and can be executed by hardware or software in the terminal.
The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet having a touch sensitive surface (e.g., a touch screen display and/or a touch pad). It should also be appreciated that in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer having a touch-sensitive surface (e.g., a touch screen display and/or a touch pad).
In the following various embodiments, a terminal including a display and a touch sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.
The execution main body of the photovoltaic panel number counting method provided by the embodiment of the application may be an electronic device or a functional module or a functional entity capable of implementing the photovoltaic panel number counting method in the electronic device, and the electronic device mentioned in the embodiment of the application includes, but is not limited to, a mobile phone, a tablet computer, a camera, a wearable device and the like.
Fig. 1 is a flowchart of a method for counting the number of photovoltaic panels according to an embodiment of the present invention, and as shown in fig. 1, the method includes:
s1, acquiring videos of areas to be counted;
it can be understood that the video of the area to be counted is obtained by shooting by adopting the camera on the unmanned aerial vehicle, specifically, according to the area position of the photovoltaic power station, the flight parameters set by the unmanned aerial vehicle are as follows: the flying height is between 200 and 300m, the flying speed is 8m/s, shooting is carried out at an angle vertical to the ground, flying is carried out in a parallel or vertical photovoltaic plate mode, the resolution of video is 1920 x 1080 or higher based on definition.
S2, selecting partial images based on each frame of video image, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
in the step, the selected target object of the partial image is marked with an external rectangular frame, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, rectangular frame information is stored, an image data set comprising the rectangular frame is obtained, and in the embodiment, the target object refers to a photovoltaic panel.
The external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, and the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information.
According to one embodiment of the invention, in a photovoltaic power plant, a rectangular frame contains photovoltaic panels and a spacing region between the photovoltaic panels, the rectangular frame of a complete photovoltaic panel is named "white_zc", the incomplete photovoltaic panel is named "half_zc", and rectangular frame information is stored in an xml file.
It should be noted that, a complete photovoltaic panel refers to two rows of panels and the edges are visible, and an incomplete photovoltaic panel refers to one row of panels and one row of frames and the edges are visible.
S3, inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
in the step, a training data set of an external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module performs feature fusion on the first feature map and outputs a second feature map, the Head module performs convolution on the second feature map, the offset of a target object prediction frame is output, and an NMS non-maximum suppression is adopted to generate a target detector of a target object.
According to one embodiment of the invention, during training, the image data of the training data set is input into a yolov5 deep learning algorithm, the algorithm reads the image data, the neural network extracts the image features, the neural network identifies according to the features, then the identification error is calculated according to the loss function, and when the error is smaller than a threshold value, model training is completed.
According to an embodiment of the present invention, taking a photovoltaic panel as a target image and taking a video image resolution of 1920×1080 as an example, the training data set preprocessing of the circumscribed rectangular frame includes: setting the image input size to 1088 x 1088, and expanding the photovoltaic panel data set by utilizing a Mosaic image enhancement technology.
It should be noted that, since yolov5 neural network downsamples the image by a factor of 32, the scaling size must be a factor of 32; because the photovoltaic panel target is larger, original pictures 1920 x 1080 scale to 1088 x 1088, key characteristic information of the photovoltaic panel cannot be lost, and meanwhile, video memory resources can be saved and training speed can be improved.
The technological process of the Mosaic: as shown in fig. 6, a gray (R channel value 114, g channel value 114, B channel value 114) base map with a size of 2176 x 2176 is first constructed, one point C (cut_x, cut_y) is piled up and selected as a splicing point in a rectangle defined by a point a (544) and a point B (1632), 4 images are randomly selected, and the regions are spliced into the base map according to the splicing points; the preprocessed data set is input to the Backbone module of the yolov5 neural network.
In accordance with one embodiment of the present invention, the yolov5 deep learning algorithm model is trained by first extracting photovoltaic panel features from a back plane module through a convolutional layer, a csp1_x structure and an SPPF structure, and outputting a first feature map of 136X 512, 68X 512, 34X 512 three scales, then, feature fusion is carried out on the three-scale first feature graphs through up-sampling and down-sampling operations in a Neck module, 128-136-68-34-512-136 second feature graphs are output, and finally, photovoltaic panel predicted frame offset parameters are output from the three second feature graphs through a 1*1 convolution kernel in a Head module
Figure SMS_50
, wherein ,/>
Figure SMS_51
、/>
Figure SMS_52
The offset of the width and the height of the photovoltaic panel prediction frame relative to the prior frame is respectively; />
Figure SMS_53
Is the confidence of the photovoltaic panel prediction frame; />
Figure SMS_54
Is a specific category of photovoltaic panel predictions, 0 is a complete photovoltaic panel, and 1 is an incomplete photovoltaic panel.
It should be noted that, the position of the photovoltaic panel prediction frame is calculated according to the following formula:
Figure SMS_55
;
Figure SMS_56
;
Figure SMS_57
;
Figure SMS_58
;
Figure SMS_61
the grid upper left corner coordinates of the central point of the photovoltaic panel prediction frame; />
Figure SMS_63
Is the center coordinate of the photovoltaic panel prediction frame relative +.>
Figure SMS_66
Offset of (2); />
Figure SMS_60
Is the center point coordinate of the photovoltaic panel prediction frame; />
Figure SMS_62
、
Figure SMS_65
The width and the height of the photovoltaic panel prediction frame are respectively; / >
Figure SMS_67
Is a Sigmoid function, limiting the predicted offset to [0,1 ]]The predicted central point does not exceed the corresponding grid area; />
Figure SMS_59
、/>
Figure SMS_64
The width and height of the a priori frame, respectively.
According to
Figure SMS_68
Value versus all photovoltaic panel predicted frame position parameters
Figure SMS_69
Screening and sorting, will->
Figure SMS_70
A box greater than 0.8 as a preselected box;
then NMS non-maximum suppression is adopted, in
Figure SMS_71
Selecting the maximum +.>
Figure SMS_72
Value +.>
Figure SMS_73
As a benchmark, calculate it and other +>
Figure SMS_74
Removing the boxes having an IOU greater than 0.85; repeating the step, selecting a new reference from the remaining preselected frames, and removing the frames with IOU greater than 0.85 until no frames with IOU greater than 0.85 are available; finally, each datum represents a complete photovoltaic panel, so that a prediction frame of the photovoltaic panel is obtained; the same is true of the acquisition step of the incomplete photovoltaic panel prediction frame.
The IOU refers to the ratio of intersection and union of a photovoltaic panel prediction frame and a real frame, and the calculation formula is as follows:
Figure SMS_75
，/>
Figure SMS_76
refers to a reference frame,/->
Figure SMS_77
Refer to other boxes, area refers to area.
Defining a loss function and training a target detector of the photovoltaic panel; the loss function is defined as the sum of the location loss, the confidence loss, and the category loss, and the calculation formula is as follows:
Figure SMS_80
wherein, K, (-) -is>
Figure SMS_83
B is the output characteristic diagram, grids and the number of anchor boxes on each grid, wherein each anchor box is a rectangular frame in which a photovoltaic panel can exist; / >
Figure SMS_85
For the weight of the corresponding item +.>
Figure SMS_79
，/>
Figure SMS_81
，/>
Figure SMS_84
；
Figure SMS_86
The kth output characteristic diagram, the ith grid and the jth anchor box are represented, if yes, the output characteristic diagram is 1, otherwise, the output characteristic diagram is 0; />
Figure SMS_78
Respectively a prediction frame and a real frame; />
Figure SMS_82
The weight of the output characteristic diagram for balancing each scale is [4.0,1.0,0.4 ]]Output feature maps corresponding to 136 x 136, 68 x 68, 34 x 34 in turn;
loss of positioning
Figure SMS_90
，
Figure SMS_94
, wherein />
Figure SMS_97
For the distance of the center point of the predicted and real frames, < >>
Figure SMS_88
For the diagonal length of the smallest bounding rectangle of the prediction and real frames, V is the aspect ratio similarity of the prediction and real frames,/o>
Figure SMS_91
For the width of the real frame +.>
Figure SMS_95
For the height of the real frame +.>
Figure SMS_98
For predicting the width of the frame, +.>
Figure SMS_87
Is the height of the prediction frame; confidence loss->
Figure SMS_92
，/>
Figure SMS_96
Prediction of frame confidence for photovoltaic panels, +.>
Figure SMS_99
IOU values for photovoltaic panels prediction and real frames,>
Figure SMS_89
for the two classification cross entropy loss->
Figure SMS_93
A weight representing a positive sample;
category loss
For the photovoltaic panel prediction frame class,
as a true frame class of photovoltaic panels,
for the purpose of the two-class cross entropy loss,
the weights representing the photovoltaic panel categories.
And judging the error of the predicted photovoltaic panel information and the real photovoltaic panel information of the neural network by analyzing the change of the result value of the loss function along with the training times, wherein the closer the predicted information is to the expected information, the smaller the loss function is, and the higher the accuracy of the target detector of the photovoltaic panel is.
S4, counting the number of target objects of each frame of image of the video based on the target detector.
In this step, as shown in fig. 2, the following specific steps are included:
s41, inputting each frame of image of the video of the region to be counted into a target detector, and obtaining apparent characteristics of a target object in each frame of image;
in the step, the video of the region to be counted is input into a target detector, the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target object
And category information collection
Coordinate information set
The expression is as follows:
Figure SMS_100
,
wherein ,
Figure SMS_101
is the t-th frame image I t A set of coordinate information of the detected target object;
Figure SMS_102
,
wherein ,
Figure SMS_103
is the coordinate information of the i-th target object, < >>
Figure SMS_104
Is the x coordinate value of the left upper corner of the circumscribed rectangle of the target object,/->
Figure SMS_105
Is the y coordinate value of the left upper corner of the circumscribed rectangle frame of the target object,/->
Figure SMS_106
Is the x coordinate value of the right lower corner of the circumscribed rectangle frame of the target object, < >>
Figure SMS_107
The y coordinate value of the right lower corner of the circumscribed rectangle frame of the target object;
category information collection
Figure SMS_108
The expression is as follows:
Figure SMS_109
,
wherein ,
Figure SMS_110
is the category of the i-th target object, +.>
Figure SMS_111
The value 0 or 1,0 represents a complete target object, and 1 represents an incomplete target object.
S42, creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
in this step, a first frame image I is displayed to the video based on the object detector 1 Detecting based on the detected coordinate information set of the target object
Figure SMS_112
Initializing and creating a Kalman tracker; kalman tracker generates a numbered set of target object coordinate information +.>
Figure SMS_113
，/>
Figure SMS_114
，/>
Figure SMS_115
Is the unique number of the ith target object, and predicts the second frame image I of the video 2 Number set of coordinate information->
Figure SMS_116
，
Figure SMS_117
，/>
Figure SMS_118
Is the predicted coordinates of the ith target object;
the calculation formula of the Kalman tracker is as follows:
Figure SMS_119
;
Figure SMS_120
,
Figure SMS_121
for the state information of the target object at time k-1, i.e.>
Figure SMS_122
, wherein ,/>
Figure SMS_123
) Is the center coordinates of the rectangular frame of the target object, s is the rectangular frame area, r is the rectangular frame aspect ratio,/->
Figure SMS_124
Is the abscissa speed, +.>
Figure SMS_125
Is the ordinate speed, +.>
Figure SMS_126
Is the area velocity;
Figure SMS_127
estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
it should be noted that, each frame of image of the video of the region to be counted is operated according to the steps.
S43, matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
In the step, the IOU distance is used as a weight by using the KM algorithm, and the t frame detection result is obtained
Figure SMS_128
And prediction result->
Figure SMS_129
Matching, when IOU is larger than or equal to 0.1, utilizing the coordinate information of the target object matched by the t-th frame
Figure SMS_130
Update->
Figure SMS_131
While continuing to predict the t+1st frame target object coordinates +.>
Figure SMS_132
The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, using +.o. of unmatched target object>
Figure SMS_133
Creating a new Kalman tracker and the number of the target object;
for the non-matched target object coordinate information
Figure SMS_134
Initializing and creating a new Kalman tracker, generating new numbers at the same time, and updating a number set of occurrence frequencies
Figure SMS_135
and />
Figure SMS_136
。
S44, counting the number sets of all the target objects qualified in matching, and calculating the number of the target objects.
In this step, the target object of each frame image is counted in sequence
Figure SMS_137
Number set of frequency of occurrence of each ID +.>
Figure SMS_138
and />
Figure SMS_139
Number set of occurrence frequency of each ID of 1 +.>
Figure SMS_140
, wherein ,/>
Figure SMS_141
The expression of (2) is as follows: />
Figure SMS_142
，/>
Figure SMS_143
Is numbered->
Figure SMS_144
The frequency of occurrence of the complete target object of (1) is the initial value, and 1 is accumulated when the complete target object occurs 1 time in one frame of image;
Figure SMS_145
the expression is as follows:
Figure SMS_146
，/>
Figure SMS_147
is numbered->
Figure SMS_148
The frequency of occurrence of incomplete target objects of (1) is the initial value, and 1 is accumulated when the incomplete target objects occur 1 time in one frame of image;
And counting the number of ID categories of which the frequency of occurrence of the target object exceeds 3 times based on the number set, obtaining the number of the target objects, and accumulating and counting to obtain the number of the target objects.
Specifically, based on a set of numbers
Figure SMS_149
Statistics of->
Figure SMS_150
ID category number more than 3, obtaining the number Clsw of the complete target object;
based on numbered sets
Figure SMS_151
Statistics of->
Figure SMS_152
ID category number more than 3, obtaining the number Clsh of incomplete target objects;
the total number of target objects M, m=clsw+clsh is counted.
In order that the above-described embodiments may be better illustrated, an example will now be described.
For example, taking a video to be detected as a flight shot video, taking a photovoltaic panel as an example, as shown in fig. 7, the flow of the photovoltaic panel number statistics method is as follows:
taking a certain photovoltaic power station as an example, determining the regional range of a photovoltaic panel in the photovoltaic power station, setting flight parameters of an unmanned aerial vehicle, setting the flight height of 200m and the flight speed of 8m/s, as shown in fig. 3, flying in the direction parallel or perpendicular to the photovoltaic panel, shooting perpendicular to the ground, and enabling the video resolution to be 1920 x 1080.
After the video is shot according to the parameters, selecting partial images and labeling external rectangular frames, wherein the rectangular frames need to comprise photovoltaic panels and interval areas between the photovoltaic panels, the name of the rectangular frames of the whole photovoltaic panels (two rows of panels and the edges of the rectangular frames are visible) is named as "white_zc", the name of the incomplete photovoltaic panels is named as "half_zc", and rectangular frame information is stored in an xml file as shown in fig. 4.
Training and learning are carried out on the training set by utilizing a yolov5 deep learning algorithm model, a target detector of the target object is generated, and the apparent characteristics of the target object are described through the generation of the target detector, as shown in fig. 5.
When the video to be detected is input, the target detector can detect the circumscribed rectangular coordinates of the photovoltaic panel in each frame of video
Figure SMS_153
And category->
Figure SMS_154
；
Figure SMS_155
，/>
Figure SMS_156
Is the t-th frame image I t A set of detected coordinate information of the photovoltaic panel;
Figure SMS_157
，/>
Figure SMS_158
is the coordinate information of the ith photovoltaic panel;
Figure SMS_159
is x coordinate value of the left upper corner of the circumscribed rectangle of the photovoltaic panel, < >>
Figure SMS_160
Is the y coordinate value of the left upper corner of the circumscribed rectangle of the photovoltaic panel, < >>
Figure SMS_161
Is x coordinate value of right lower corner of the rectangle circumscribed by the photovoltaic panel, < >>
Figure SMS_162
The y coordinate value of the right lower corner of the circumscribed rectangle of the photovoltaic panel; />
Figure SMS_163
，
Figure SMS_164
Is a category of i-th photovoltaic panel, < >>
Figure SMS_165
The value 0 or 1,0 represents a complete photovoltaic panel, and 1 represents an incomplete photovoltaic panel;
first frame photovoltaic panel image I with target detector 1 Detecting, using the detected coordinate information set of the photovoltaic panel
Figure SMS_166
Initializing and creating a Kalman tracker; kalman tracker generates a set of photovoltaic panel numbers
Figure SMS_167
，/>
Figure SMS_168
，/>
Figure SMS_169
Is the unique number of the ith photovoltaic panel, and predicts the second frame of photovoltaic panel image I 2 Photovoltaic panel coordinate information->
Figure SMS_170
，/>
Figure SMS_171
，
Figure SMS_172
Is the predicted coordinates of the ith photovoltaic panel;
Statistics I 1 In (a)
Figure SMS_173
Frequency of occurrence of each ID>
Figure SMS_174
and />
Figure SMS_175
1 frequency of occurrence of the respective IDs +.>
Figure SMS_176
，
Figure SMS_177
，/>
Figure SMS_178
Is numbered->
Figure SMS_179
The frequency of occurrence of the complete photovoltaic panel is 1, and 1 is accumulated every 1 occurrence in 1 frame of image;
Figure SMS_180
，/>
Figure SMS_181
is numbered->
Figure SMS_182
The frequency of occurrence of incomplete photovoltaic panels, the initial value is 1, and 1 is accumulated every 1 occurrence in 1 frame of image;
detecting the subsequent image by using the target detector, and detecting the coordinate information of the photovoltaic panel detected by the t-th frame image
Figure SMS_183
And predictive information +.>
Figure SMS_184
；
Using KM algorithm, taking IOU distance as weight, and detecting the t frame
Figure SMS_185
And prediction result
Figure SMS_186
Matching is carried out; based on preset unmanned aerial vehicle flight parameters and actual flight conditions, the IOU distance threshold is set to be 0.1, and when the IOU is less than 0.1, the unmanned aerial vehicle is not considered to be the same target;
photovoltaic panel coordinate information matched by using t-th frame
Figure SMS_187
Updating a Kalman tracker with simultaneous relayContinuously predicting coordinates of photovoltaic panel in t+1th frame +.>
Figure SMS_188
；
For unmatched photovoltaic panels
Figure SMS_189
A new kalman tracker should be initialized and created while generating new numbers, updating the set of numbers +.>
Figure SMS_190
and />
Figure SMS_191
；
According to the number set
Figure SMS_192
Statistics of->
Figure SMS_193
Obtaining the number Clsw of the complete photovoltaic panels according to ID category number more than 3; />
According to the number set
Figure SMS_194
Statistics of->
Figure SMS_195
Obtaining the number Clsh of incomplete photovoltaic panels according to ID category numbers more than 3;
Finally, photovoltaic panel number = clsw+clsh/2.
According to one embodiment of the invention, in the photovoltaic panel quantity counting device, the acquisition module shoots videos obtained from the area to be counted by adopting a camera on the unmanned aerial vehicle, specifically, according to the area position of the photovoltaic power station, the flight parameters set by the unmanned aerial vehicle are as follows: the flying height is between 200 and 300m, the flying speed is 8m/s, shooting is carried out at an angle vertical to the ground, flying is carried out in a parallel or vertical photovoltaic plate mode, the resolution of video is 1920 x 1080 or higher based on definition.
The training set making module is used for selecting partial images based on each frame of image of the video, and making a training data set by marking a target object in the selected images with an external rectangular frame.
According to one embodiment of the invention, in a photovoltaic power plant, a rectangular frame contains photovoltaic panels and a spacing region between the photovoltaic panels, the rectangular frame of a complete photovoltaic panel is named "white_zc", the incomplete photovoltaic panel is named "half_zc", and rectangular frame information is stored in an xml file.
It should be noted that, a complete photovoltaic panel refers to two rows of panels and the edges are visible, and an incomplete photovoltaic panel refers to one row of panels and one row of frames and the edges are visible.
And the training module inputs the training data set into a yolov5 deep learning algorithm model to perform training learning, generates a target detector of the target object, and describes the apparent characteristics of the target object by generating the target detector.
According to one embodiment of the invention, during training, the image data of the training set is input into a yolov5 deep learning algorithm, the algorithm reads the image data, the neural network extracts the image features, the neural network performs recognition according to the features, then calculates recognition errors according to a loss function, and completes model training when the errors are smaller than a threshold value.
And the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
The photovoltaic panel number counting device in the embodiment of the application can be an electronic device, and also can be a component in the electronic device, such as an integrated circuit or a chip. The electronic device may be a terminal, or may be other devices than a terminal. By way of example, the electronic device may be a mobile phone, tablet computer, notebook computer, palm computer, vehicle-mounted electronic device, mobile internet appliance (Mobile Internet Device, MID), augmented reality (augmented reality, AR)/Virtual Reality (VR) device, robot, wearable device, ultra-mobile personal computer, UMPC, netbook or personal digital assistant (personal digital assistant, PDA), etc., but may also be a server, network attached storage (Network Attached Storage, NAS), personal computer (personal computer, PC), television (TV), teller machine or self-service machine, etc., and the embodiments of the present application are not limited in particular.
The photovoltaic panel number statistics device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which is not specifically limited in the embodiments of the present application.
The photovoltaic panel number statistics device provided in the embodiment of the present application can implement each process implemented by the method embodiments of fig. 1 to 2, and in order to avoid repetition, a detailed description is omitted here.
In some embodiments, as shown in fig. 8, the embodiment of the present application further provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and capable of running on the processor 701, where the program when executed by the processor 701 implements the respective processes of the above embodiment of the method for counting the number of photovoltaic panels, and the same technical effects can be achieved, and for avoiding repetition, a detailed description is omitted herein.
The electronic device in the embodiment of the application includes the mobile electronic device and the non-mobile electronic device described above.
The invention also provides a computer readable storage medium, wherein the storage medium stores a computer program, and the computer program is set to realize the method for counting the number of the photovoltaic panels according to the embodiment of the invention when the computer program runs.
It should be noted that the logic and/or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a ordered listing of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the device or element being referred to must have a specific orientation, be configured and operated in a specific orientation, and therefore should not be construed as limiting the present invention.
Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In the description of the present invention, the meaning of "plurality" means at least two, for example, two, three, etc., unless specifically defined otherwise.
In the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," "secured," and the like are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally formed; can be mechanically or electrically connected; either directly or indirectly, through intermediaries, or both, may be in communication with each other or in interaction with each other, unless expressly defined otherwise. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
In the present invention, unless expressly stated or limited otherwise, a first feature "up" or "down" a second feature may be the first and second features in direct contact, or the first and second features in indirect contact via an intervening medium. Moreover, a first feature being "above," "over" and "on" a second feature may be a first feature being directly above or obliquely above the second feature, or simply indicating that the first feature is level higher than the second feature. The first feature being "under", "below" and "beneath" the second feature may be the first feature being directly under or obliquely below the second feature, or simply indicating that the first feature is less level than the second feature.
While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.

Claims (10)

1. A method for counting the number of photovoltaic panels, comprising the steps of:
acquiring a video of an area to be counted;
selecting partial images based on each frame of image of the video, and labeling a target object in the selected images with an external rectangular frame to manufacture a training data set;
inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image;
the number of target objects in each frame of image of the video is counted based on the target detector.
2. The method according to claim 1, wherein selecting a partial image based on each frame of video image, labeling a target object in the selected image with an external rectangular frame, and creating a training data set, includes:
selecting partial images based on each frame of images of the video, labeling an external rectangular frame on a target object in the images, and manufacturing a training data set, wherein the external rectangular frame is based on LabelImg image labeling software, the external rectangular frame of the target object is manually labeled, the LabelImg image labeling software automatically generates an xml format file and stores rectangular frame information, the rectangular frame comprises the target object and a spacing area between any two adjacent target objects, and the rectangular frame information is stored.
3. The method according to claim 1, wherein inputting the training data set of the circumscribed rectangle frame into a deep learning algorithm model for training to obtain a target detector capable of describing apparent characteristics of a target object in each frame of image comprises:
the training data set of the external rectangular frame is preprocessed and then input into a yolov5 deep learning algorithm model for training learning, the yolov5 deep learning algorithm model comprises a backbox module, a Neck module and a Head module, the backbox module extracts features and outputs a first feature map, the Neck module carries out feature fusion on the first feature map and outputs a second feature map, the Head module carries out convolution on the second feature map, the offset of a target object prediction frame is output, and the target detector of the target object is generated by adopting NMS non-maximum suppression.
4. The method for counting the number of photovoltaic panels according to claim 1, wherein the counting the number of target objects in each frame of image of the video based on the target detector comprises:
inputting each frame of video image of the region to be counted into a target detector, and obtaining the apparent characteristics of a target object in each frame of image;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, and predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker;
Matching the detection result of the apparent characteristics of any frame of target object with the prediction result;
and counting the number set of each target object qualified in matching, and calculating the number of the target objects.
5. The method for counting the number of the photovoltaic panels according to claim 4, wherein inputting each frame of image of the video of the area to be counted into the target detector, and obtaining the apparent characteristics of the target object in each frame of image, comprises:
inputting the video of the region to be counted into a target detector, wherein the target detector detects the apparent characteristics of the circumscribed rectangular frame of the target object in each frame of video, and the apparent characteristics comprise a coordinate information set of the circumscribed rectangular frame of the target object
Figure QLYQS_1
And category information set->
Figure QLYQS_2
Coordinate information set->
Figure QLYQS_3
The expression is as follows:
Figure QLYQS_4
；
wherein ,
Figure QLYQS_5
is the t-th frame image I t A set of coordinate information of the detected target object; />
Figure QLYQS_6
；
wherein ,
Figure QLYQS_7
is the coordinate information of the i-th target object, < >>
Figure QLYQS_8
Is the x coordinate value of the left upper corner of the circumscribed rectangle of the target object,/->
Figure QLYQS_9
Is the y coordinate value of the left upper corner of the circumscribed rectangle frame of the target object,/->
Figure QLYQS_10
Is the x coordinate value of the right lower corner of the circumscribed rectangle frame of the target object, < >>
Figure QLYQS_11
The y coordinate value of the right lower corner of the circumscribed rectangle frame of the target object;
category information collection
Figure QLYQS_12
The expression is as follows:
Figure QLYQS_13
；
wherein ,
Figure QLYQS_14
is the category of the i-th target object, +.>
Figure QLYQS_15
The value 0 or 1,0 represents a complete target object, and 1 represents an incomplete target object;
creating a Kalman tracker and a number set of the target object based on the apparent characteristics of the target object, predicting the apparent characteristics of the target object in the next frame of image based on the Kalman tracker, comprising:
video first frame image I based on object detector 1 Detecting based on the detected coordinate information set of the target object
Figure QLYQS_16
Initializing and creating a Kalman tracker; kalman tracker generates a numbered set of target object coordinate information +.>
Figure QLYQS_17
，/>
Figure QLYQS_18
，/>
Figure QLYQS_19
Is the unique number of the ith target object, and predicts the second frame image I of the video 2 Number set of coordinate information->
Figure QLYQS_20
，
Figure QLYQS_21
，/>
Figure QLYQS_22
Is the predicted coordinates of the ith target object;
the calculation formula of the Kalman tracker is as follows:
Figure QLYQS_23
；
Figure QLYQS_24
；
Figure QLYQS_25
for the state information of the target object at time k-1, i.e.>
Figure QLYQS_26
, wherein ,/>
Figure QLYQS_27
) Is the center coordinates of the rectangular frame of the target object, s is the rectangular frame area, r is the rectangular frame aspect ratio,/->
Figure QLYQS_28
Is the abscissa speed, +.>
Figure QLYQS_29
Is the ordinate speed, +.>
Figure QLYQS_30
Is the area velocity;
Figure QLYQS_31
estimating a covariance matrix for the state of the target object at the moment k-1, wherein A is a state transition matrix, and Q is a process noise covariance matrix;
Each frame of image of the video of the area to be counted is operated according to the steps;
matching the detection result of the apparent characteristic of any frame target object with the prediction result, comprising:
using KM algorithm, taking IOU distance as weight, and detecting the t frame
Figure QLYQS_32
And prediction result->
Figure QLYQS_33
Matching, when IOU is more than or equal to 0.1, using target object coordinate information matched by t frame +.>
Figure QLYQS_34
Update->
Figure QLYQS_35
While continuing to predict the t+1st frame target object coordinates +.>
Figure QLYQS_36
The method comprises the steps of carrying out a first treatment on the surface of the When IOU is less than 0.1, using +.o. of unmatched target object>
Figure QLYQS_37
Creating a new Kalman tracker and the number of the target object;
counting the number set of each target object qualified by matching, and calculating the number of the target objects, wherein the number set comprises the following steps:
sequentially counting target objects of each frame of image
Figure QLYQS_38
Numbering sets of individual ID occurrences
Figure QLYQS_39
and />
Figure QLYQS_40
Number set of occurrence frequency of each ID of 1 +.>
Figure QLYQS_41
, wherein ,
Figure QLYQS_42
the expression of (2) is as follows:
Figure QLYQS_43
，
Figure QLYQS_44
is numbered->
Figure QLYQS_45
The frequency of occurrence of the complete target object of (1) is the initial value, and 1 is accumulated when the complete target object occurs 1 time in one frame of image;
Figure QLYQS_46
the expression is as follows:
Figure QLYQS_47
，
Figure QLYQS_48
is numbered->
Figure QLYQS_49
The frequency of occurrence of incomplete target objects of (1) is the initial value, and 1 is accumulated when the incomplete target objects occur 1 time in one frame of image;
and counting the number of ID categories of which the frequency of occurrence of the target object exceeds 3 times based on the number set, obtaining the number of the target objects, and accumulating and counting to obtain the number of the target objects.
6. The method according to claim 5, wherein for the unmatched target object coordinate information
Figure QLYQS_50
Initializing and creating a new Kalman tracker, generating new numbers at the same time, and updating the number set of occurrence frequency +.>
Figure QLYQS_51
and />
Figure QLYQS_52
。
7. The method according to claim 5, wherein counting the number of ID categories of the target object occurring more than 3 times based on the number set, obtaining the number of target objects, and accumulating the statistics to obtain the number of target objects, comprises:
based on numbered sets
Figure QLYQS_53
Statistics of->
Figure QLYQS_54
ID category number more than 3, obtaining the number Clsw of the complete target object;
based on numbered sets
Figure QLYQS_55
Statistics of->
Figure QLYQS_56
ID category number more than 3, obtaining the number Clsh of incomplete target objects;
the total number of target objects M, m=clsw+clsh is counted.
8. A photovoltaic panel count apparatus, comprising:
the acquisition module is used for acquiring videos of the areas to be counted;
the training set making module is used for selecting partial images based on each frame of image of the video, labeling a target object in the selected images with an external rectangular frame, and making a training data set;
the training module is used for inputting the training data set of the circumscribed rectangular frame into a deep learning algorithm model for training so as to obtain a target detector capable of describing the apparent characteristics of a target object in each frame of image;
And the statistics module is used for counting the number of target objects of each frame of image of the video based on the target detector.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method of counting the number of photovoltaic panels according to any one of claims 1-7 when executing the program.
10. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method of counting the number of photovoltaic panels according to any one of claims 1-7.
CN202310420646.4A 2023-04-19 2023-04-19 Photovoltaic panel quantity counting method and device, electronic equipment and storage medium Pending CN116128883A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310420646.4A CN116128883A (en) 2023-04-19 2023-04-19 Photovoltaic panel quantity counting method and device, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310420646.4A CN116128883A (en) 2023-04-19 2023-04-19 Photovoltaic panel quantity counting method and device, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116128883A true CN116128883A (en) 2023-05-16

Family

ID=86312201

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310420646.4A Pending CN116128883A (en) 2023-04-19 2023-04-19 Photovoltaic panel quantity counting method and device, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116128883A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117132419A (en) * 2023-10-26 2023-11-28 北京图知天下科技有限责任公司 Numbering method of photovoltaic module
CN117611929A (en) * 2024-01-23 2024-02-27 湖北经济学院 LED light source identification method, device, equipment and medium based on deep learning
CN117611929B (en) * 2024-01-23 2024-04-23 湖北经济学院 LED light source identification method, device, equipment and medium based on deep learning

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106127812A (en) * 2016-06-28 2016-11-16 中山大学 A kind of passenger flow statistical method of non-gate area, passenger station based on video monitoring
US20190304102A1 (en) * 2018-03-30 2019-10-03 Qualcomm Incorporated Memory efficient blob based object classification in video analytics
WO2020207038A1 (en) * 2019-04-12 2020-10-15 深圳壹账通智能科技有限公司 People counting method, apparatus, and device based on facial recognition, and storage medium
CN111860282A (en) * 2020-07-15 2020-10-30 中国电子科技集团公司第三十八研究所 Subway section passenger flow volume statistics and pedestrian retrograde motion detection method and system
CN113469955A (en) * 2021-06-15 2021-10-01 车嘉祺 Photovoltaic module fault area image detection method and system
KR20220051055A (en) * 2020-10-16 2022-04-26 이지스로직 주식회사 Solar panel inspection device using drone
CN114677554A (en) * 2022-02-25 2022-06-28 华东理工大学 Statistical filtering infrared small target detection tracking method based on YOLOv5 and Deepsort
WO2022134120A1 (en) * 2020-12-26 2022-06-30 西安科锐盛创新科技有限公司 Target motion prediction-based parking lot management and control method, apparatus, and electronic device

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106127812A (en) * 2016-06-28 2016-11-16 中山大学 A kind of passenger flow statistical method of non-gate area, passenger station based on video monitoring
US20190304102A1 (en) * 2018-03-30 2019-10-03 Qualcomm Incorporated Memory efficient blob based object classification in video analytics
WO2020207038A1 (en) * 2019-04-12 2020-10-15 深圳壹账通智能科技有限公司 People counting method, apparatus, and device based on facial recognition, and storage medium
CN111860282A (en) * 2020-07-15 2020-10-30 中国电子科技集团公司第三十八研究所 Subway section passenger flow volume statistics and pedestrian retrograde motion detection method and system
KR20220051055A (en) * 2020-10-16 2022-04-26 이지스로직 주식회사 Solar panel inspection device using drone
WO2022134120A1 (en) * 2020-12-26 2022-06-30 西安科锐盛创新科技有限公司 Target motion prediction-based parking lot management and control method, apparatus, and electronic device
CN113469955A (en) * 2021-06-15 2021-10-01 车嘉祺 Photovoltaic module fault area image detection method and system
CN114677554A (en) * 2022-02-25 2022-06-28 华东理工大学 Statistical filtering infrared small target detection tracking method based on YOLOv5 and Deepsort

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
LBJ_WZ: "Deepsort源码详解及个人理解", 《HTTPS://BLOG.CSDN.NET/QQ_42779673/ARTICLE/DETAILS/116950649》, pages 1 - 16 *
TAO HONG等: "A Real-Time Tracking Algorithm for Multi-Target UAV Based on Deep Learning", 《MDPI》, pages 1 - 18 *
李广: "基于视频图像的车流量的检测研究", 《中国优秀硕士学位论文全文数据库（电子期刊）》, pages 17 - 32 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117132419A (en) * 2023-10-26 2023-11-28 北京图知天下科技有限责任公司 Numbering method of photovoltaic module
CN117132419B (en) * 2023-10-26 2024-01-23 北京图知天下科技有限责任公司 Numbering method of photovoltaic module
CN117611929A (en) * 2024-01-23 2024-02-27 湖北经济学院 LED light source identification method, device, equipment and medium based on deep learning
CN117611929B (en) * 2024-01-23 2024-04-23 湖北经济学院 LED light source identification method, device, equipment and medium based on deep learning

Similar Documents

Publication Publication Date Title
US10210418B2 (en) Object detection system and object detection method
CN109829398B (en) Target detection method in video based on three-dimensional convolution network
US8462987B2 (en) Detecting multiple moving objects in crowded environments with coherent motion regions
JP5181704B2 (en) Data processing apparatus, posture estimation system, posture estimation method and program
CN110309842B (en) Object detection method and device based on convolutional neural network
CN108182695B (en) Target tracking model training method and device, electronic equipment and storage medium
CN116152863B (en) Personnel information identification method and device, electronic equipment and storage medium
CN114926747A (en) Remote sensing image directional target detection method based on multi-feature aggregation and interaction
US20170053172A1 (en) Image processing apparatus, and image processing method
CN116128883A (en) Photovoltaic panel quantity counting method and device, electronic equipment and storage medium
Zhang et al. PSNet: Perspective-sensitive convolutional network for object detection
CN116824335A (en) YOLOv5 improved algorithm-based fire disaster early warning method and system
CN115690545A (en) Training target tracking model and target tracking method and device
US11836960B2 (en) Object detection device, object detection method, and program
CN117132649A (en) Ship video positioning method and device for artificial intelligent Beidou satellite navigation fusion
CN112926681B (en) Target detection method and device based on deep convolutional neural network
CN112541403B (en) Indoor personnel falling detection method by utilizing infrared camera
CN116052097A (en) Map element detection method and device, electronic equipment and storage medium
CN115205806A (en) Method and device for generating target detection model and automatic driving vehicle
CN114067359A (en) Pedestrian detection method integrating human body key points and attention features of visible parts
JP2022531029A (en) Image recognition method, device and storage medium
CN117523428B (en) Ground target detection method and device based on aircraft platform
CN113177545B (en) Target object detection method, target object detection device, electronic equipment and storage medium
KR20200005853A (en) Method and System for People Count based on Deep Learning
CN108490930A (en) A kind of good intelligent robot of navigation performance

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20230516