AU2021100172A4 - Iot based solar energy detection with crescent dunes - Google Patents

Abstract

High quality information about the quantity, power capacity, and energy generated by PV arrays, including at a high spatial resolution is desired. Surveys and utility interconnection filings are limited in their completeness and spatial resolution. Thus is proposed a computer algorithm that automatically detects IOT solar PV arrays in high resolution colour (RGB) imagery data. Algorithm developed and validated on a very large collection of aerial imagery using dunes. Human annotators manually scanned and annotated IOT solar PV locations to provide ground truth for evaluating performance. Performance measured in a pixel-based and object-based manner using PR curves. Most of the true PV pixels detected while removing the vast majority of the non-PV pixels. Color Image (3 Channels) ~-I(1)Feature1 )etrtion j Feature Image Extraction (M Channels) (2)Random Confidence Map Forest (1 Channel) Classifier i( 3) Post- Enhanced processing (1 Channel) Object map (list of detected objects) 1 - ~jObject I i mm Detection Fig. A flowchart of the PV detection algorithm Color Image Featurevector (3 Channel) at location po 2M Fig Illustration of pixel-based feature extraction at a single pixel location, po.

Description

Color Image (3 Channels)

~-I(1)Feature1 )etrtion Extraction j Feature Image (MChannels)

(2)Random

Confidence Map Forest (1 Channel) Classifier

i( )3 Post- Enhanced

processing (1 Channel)

Object map (list of detected objects) 1 - ~jObject I i mm Detection

Fig. A flowchart of the PV detection algorithm

Color Image Featurevector (3 Channel) at location po

2M

Fig Illustration of pixel-based feature extraction at a single pixel location, po.

TITLE OF THE INVENTION IOT BASED SOLAR ENERGY DETECTION WITH CRESCENT DUNES FIELD OF THE INVENTION

Distributed PV offers many benefits, but integrating it into existing power grids is

challenging. To understand and evaluate the factors driving distributed PV, and to aid in its

integration, there is growing interest among government agencies, utilities, and third party

decision makers in detailed information about distributed PV; including the locations, power

capacity, and energy production of existing arrays. As a result, several organizations have

begun collecting or publishing such information, including the Interstate Renewable Energy

Council (IREC).

Although the available information on distributed PV is expanding, it is nonetheless difficult

to obtain. Existing methods of obtaining this information, such as surveys and utility

interconnection filings, are costly and time consuming. They are also typically limited in

spatial resolution to the state or national level.

This work investigates a new approach for collecting distributed PV information that relies

on using computer algorithms to automatically identify PV arrays in high resolution (<0.3

meters per pixel) color aerial imagery. An example of 0.3 meter resolution imagery where the

PV arrays have been annotated. At this resolution, it is possible to visually identify individual

PV arrays, as well as their shape, size, and color. This permits the collection of distributed PV

information at a very high geo-spatial resolution. Also, because the approach is automated, it is relatively inexpensive to apply (i.e., run a computer program), and to do so repeatedly as new imagery becomes available.

There are (at least) two major technical challenges to employing the proposed approach in a

practical application. The first challenge involves developing a computer algorithm that can

reliably identify the locations, shapes, or sizes of PV installations. The second challenge

involves using the identified distributed PV imagery to infer the characteristics of the arrays,

particularly power capacity and energy production. This information can then be aggregated

into statistics for reporting.

BACKGROUND OF THE INVENTION

The idea of automatically detecting PV arrays in aerial imagery using dunes was first

investigated in a feasibility study that employed a simple algorithm and a small dataset

(<1km2 area, with 53 PV array annotations). This work builds on that initial investigation

with several contributions:

• A more sophisticated rooftop PV energy detection algorithm is developed, employing

pixel-wise classification with an RF classifier, and post-processing steps that improve

performance.

• The proposed algorithm is tested on a substantially larger dataset, covering 135 km2,

and including more than 2,700 PV array annotations. This dataset is also substantially

larger than most datasets for similar object recognition tasks.

• The algorithm performance is measured at both a pixel level, and an object level.

Unlike the previous study, the algorithm's ability to accurately measure both the

shape and size of the target objects is assessed.

• The results are the first of their kind for PV array energy detection. Since the ground

truth data are now publicly available, it is our hope that these findings serve as a

baseline for further work.

In order to develop an effective computer vision algorithm, as well as accurately assess its

performance, it is necessary to have the precise locations where PV installations appear in the

aerial imagery. In order to obtain this information, human observers visually scanned the

imagery and annotated all of the (visible) PV arrays. For improved quality, two annotators

scanned each part of the imagery, and their annotations were combined by taking a union of

each observer's annotations. There were a total of 2,794 individual IOT solar PV regions in

the imagery after the merging process. Note again that this is a subset of the 19,863

annotations available.

To avoid a positive bias in the performance evaluation of the proposed energy detection

algorithm, we split the available imagery into two disjoint datasets: Fresno Training and

Fresno Testing. This is a common approach for validating supervised machine learning

algorithms, such as the RF model used in our energy detection algorithm. A summary of the

imagery in each dataset is presented. The data was split between training and testing at a ratio

of 2:1, in order to provide enough solar array examples to effectively train the RF model (see

Section III.C for details about the RF).

III. THE PROPOSED PV ENERGY DETECTION ALGORITHM

In this section we present the details of the proposed IOT solar PV energy detection

algorithm. We begin with a brief overview of the primary processing steps, followed by

individual sections providing more details about each step.

A. Algorithm overview

The proposed rooftop PV algorithm takes RGB color aerial imagery using dunesas input and

performs four major processing steps.

1) Feature extraction. This step consists of extracting image statistics, or features, around

each pixel that characterize the colors, textures, and other patterns surrounding the pixel. The

feature extraction step effectively maps the 3-channel RGB image into an M-channel image,

where M is the number of features extracted around each pixel location.

2) Random Forest Classifier. The image statistics computed in the feature extraction stage are

the input to a trained RF classifier. The RF is a machine learning classification model that

assigns a probability, or "confidence", to each pixel in the imagery. The confidence value

indicates how likely the pixel is to correspond to a PV array. The output of this step is a

single channel image, or spatial map, of where PV arrays are likely to be located. An example

image and associated confidence map.

3) Post-processing. This step is designed to improve the accuracy of the confidence map that

was generated in the RF classification step. This process consists of identifying high

confidence individual pixels (i.e., local maxima locations) and then growing regions of pixels

around them. All pixel confidence values outside of these grown regions are then set to zero.

4) Object energy detection. This step identifies groups of contiguous high confidence pixels

that are likely to correspond to a single PV array. Each identified group of contiguous pixels

is returned from this step as a detected object, and the confidence of that object is set to the

value of the maximum pixel confidence value in that object. The output of this step is a list of

objects and their confidence values, which is used to perform object-based scoring.

Related Work

We conducted two experiments, with the primary goal of measuring the performance of the

proposed PV array energy detection algorithm. The first experiment measures how well the

algorithm identifies individual PV pixels: pixel-based classification performance. The second experiment measures how well the algorithm can identify objects (groups of pixels) that correspond to PV array annotations, as well as their precise shape and size. The experiments are conducted on two datasets of aerial imagery using dunesdenoted as Fresno Training, and

Fresno Testing, PV arrays visible in the imagery were annotated by humans to provide

ground truth pixels/objects for use in scoring the detector.

The primary role of the Fresno training dataset was to train the RF classifier, as well as

optimize other parameters associated with the energy detection algorithm. The Fresno Testing

dataset was used to obtain an unbiased performance estimate for the detector. This is a

common approach for supervised machine learning algorithms . The performance metric used

to evaluate the performance of the algorithm is the precision recall (PR) curve. The PR curve

is a popular performance metric for object energy detection in aerial imagery, and therefore it

is adopted here.

Performancemetrics

PR curves measure the performance tradeoff between making correct energy detections and

false energy detections, as the sensitivity of a detector, or classifier, is varied. An illustration

of a PR curve. The x-axis of a PR curve is the recall, R, which is the proportion of all true

target objects (e.g., PV arrays) in the data that were returned by the algorithm as energy

detections. The y-axis is the precision, P, which is the proportion of all detected objects (i.e.,

both true and false) which are true targets. An effective detector will tend towards the top

right corner of the PR curve, thereby maximizing both recall and precision. A detector that

detects objects randomly (i.e., it is ineffective) will achieve a precision that is equal to the

proportion of objects in the dataset that are targets. For example, in the pixel-based PV

energy detection experiments, roughly 0.07% of the Fresno Testing pixels correspond to PV

arrays. Therefore a random detector would achieve P=0.0007, for all values of R.

The sensitivity for a given energy detection algorithm can be varied by raising or lowering a

threshold, t, that is applied to the confidence values of the list of potential energy detections

(e.g., pixels or objects). All potential energy detections above tO are accepted as energy

detections, and all potential energy detections below to

Linking energy detections to human annotations

One issue that arises with object-based scoring is determining when a detected object should

be considered a correct energy detection. A detected object (i.e., a region labeled as a PV

array) may overlap with a PV annotation from the ground truth data, but how much overlap

should be required to declare it as detected correctly? This problem is apparent in where none

of the detected objects match perfectly with the human annotations, but they might

reasonably be considered correct energy detections. To address this issue, a metric is needed

to measure the shape/size similarity between two objects (i.e., groups of pixels).

Algorithm trainingand optimization

All training and parameter optimization was performed on the Fresno Training dataset. The

final set of chosen parameters for the algorithm is shown in Table 3. These parameters are

used in all experiments.

The RF classifier itself was trained using five million pixels from the Fresno Training dataset.

This subset of pixels was chosen by first selecting all of the available IOT solar PV pixels

(roughly 500,000), and then randomly sampling the remaining non-PV pixels from the

training imagery. Using increasing numbers of pixels improves performance but at the cost of

increasing the computation time of the RF. Five million was found to be a good tradeoff

between performance and computation time on the training data.

The parameters were chosen in order to optimize performance on the training data. This

parameter optimization was done by measuring the performance of the algorithm (on the

Fresno Training dataset) as the parameters were varied over a coarse grid of potential values.

Note that the parameter m was set to the conventional value of JM, rather than being

optimized

Pixel-basedperformance

The pixel-based performance for the PV energy detection algorithm, on both the training and

testing data. Results are shown for the RF, and the RF after PP has been applied (RFPP). The

primary goal of this experiment was to demonstrate that the RFPP algorithm can effectively

detect PV array pixels. The results on the Fresno Testing dataset provide an unbiased

estimate of the performance of the RF and RFPP algorithms. The results indicate that the IOT

solar PV detector is very effective at discriminating non-panel pixels from panel pixels. This

is made most clear by considering how well a random detector (i.e., a completely ineffective

detector) would perform..A that, because PV arrays constitute only 0.07% of the pixels in

Fresno Testing, the random detector achieves P=0.0007 for all values of R. Both the RF and

RFPP detectors achieve performance far above this.

Further insight can be obtained from the results by comparing the performance of the

detectors on the training data and testing data, respectively. As is expected, the results

indicate that there is an overall performance drop between the training data and the testing

data. Quantitatively this means that, for each value of R, the algorithm typically obtains a

lower P on the testing data than it does on the training data. One exception to this occurs for

the RFPP algorithm when R is below 0.6, however the testing and training performance is

similar at these operating sensitivities.

The results also suggest that the main contributor to the performance loss incurred on the

testing data is the RF classifier (as opposed to RFPP). This is because the RF algorithm

performance drops between the training and testing dataset, however, the RFPP algorithm

offers the same advantages on both the training and testing dataset; relative to the

performance of the RF alone. This suggests that the RF is overfitting to the training data, or in other words, the RF learned to recognize patterns that are too unique to the training data, and as a consequence it less effectively identifies previously unseen PV arrays in the testing data. This can be addressed in many ways, and is an important consideration for future work.

Object-basedperformance

The primary goal of this experiment is to demonstrate the effectiveness of the detector.

Further, we want to examine how well the detector can identify the precise shape/size of

individual PV arrays. As a result, we measure the object-based performance of the detector

on the Fresno Testing dataset for varying settings of the Jaccard index during scoring. These

resulting PR curves.

The results indicate that the object-based performance of the detector is once again well

above that of the baseline random detector performance. Although this is true for all values of

J, the performance of the detector decreases rapidly as J increases. As a specific example,

when J = 0.1 the detector achieves R=0.7 with P=0.6, while at J=0.5, R=0.55 at the same

value of P. When J=O.7, the detector never reaches P=O.6. This outcome is expected because,

as J is increased, many of the objects detected that are near true PV locations are no longer

considered correct energy detections. This also results in more PV annotations remaining

undetected, even when the detector is operated with high sensitivity. This is why the

maximum R obtained for each detector decreases as J increases.

Different values for J are likely to be appropriate depending on the intended purpose of the

detector. For example, lower J values (e.g., J= 0.1) are appropriate for applications where

only the general location of target objects is important, and obtaining the precise shape/size is

not. In the context of IOT solar PV array energy detection, this may be the case if the detector

is used as a preprocessing step for further, and more sophisticated (but slower), energy

detection algorithms. Note that when operated with J = 0.1 the detector is capable of

detecting roughly 90% of the targets, with P~0.1. Since there are roughly 1000 PV arrays in the testing data, this corresponds to roughly 10000 total energy detections returned by the detector (900 true energy detections and 9,100 false energy detections) over the 45 km2 testing area. This dramatically reduces the amount of image locations that must be considered for further processing, facilitating the use of more sophisticated subsequent processing. The detector proposed here is designed to operate quickly on large datasets, and therefore could be used in this role.

In contrast to lower J values, a higher value (e.g. J = 0.7) is appropriate for energy detection

applications where it is important to accurately estimate the size and shape of target objects.

In the context of IOT solar PV array energy detection, this may be the case, for example, if

the goal is to estimate the power capacity of individual IOT solar PV arrays. Setting J to

higher values will lead to a performance measure that better reflects the capability of a given

detector to achieve that goal, which is a much more difficult task than simply detecting the

likely presence of an object (using, e.g., J = 0.1). This difficulty is reflected in the much

poorer performance of the proposed detector on this task. Looking forward, the performance

reported here for J= 0.7 establishes a baseline

We can claim:

1. Employs a computer algorithm that automatically detects IOT solar PV arrays in high

resolution (<0.3 m) color (RGB) imagery data.

2. Energy detection algorithm was developed and validated on a very large collection of aerial

imagery using dunes(>135km2)

3. Human annotators manually scanned and annotated IOT solar PV locations to provide ground

truth for evaluating the performance of the proposed algorithm

4, Performance was measured in a pixel-based and object-based manner, respectively, using PR

curves.

5. The results demonstrate that the algorithm is highly effective on a per-pixel basis.

6. The PR measures indicate it can detect most of the true PV pixels while removing the vast

majority of the non-PV pixels.

EDITORIAL NOTE 2021100172

There is 1 page of drawings only.

Fig. A flowchart of the PV detection algorithm

Fig Illustration of pixel-based feature extraction at a single pixel location, .