WO2014021996A1 - System and method for reduced incremental spectral clustering - Google Patents
System and method for reduced incremental spectral clustering Download PDFInfo
- Publication number
- WO2014021996A1 WO2014021996A1 PCT/US2013/045422 US2013045422W WO2014021996A1 WO 2014021996 A1 WO2014021996 A1 WO 2014021996A1 US 2013045422 W US2013045422 W US 2013045422W WO 2014021996 A1 WO2014021996 A1 WO 2014021996A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- clusters
- cluster
- basis vector
- image data
- coefficients
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims abstract description 147
- 230000003595 spectral effect Effects 0.000 title claims abstract description 28
- 239000013598 vector Substances 0.000 claims abstract description 209
- 230000009467 reduction Effects 0.000 description 35
- 238000012545 processing Methods 0.000 description 18
- 238000007906 compression Methods 0.000 description 14
- 230000006835 compression Effects 0.000 description 14
- 230000002547 anomalous effect Effects 0.000 description 13
- 239000000463 material Substances 0.000 description 12
- 238000001228 spectrum Methods 0.000 description 10
- 238000004458 analytical method Methods 0.000 description 8
- 238000000701 chemical imaging Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 4
- 230000008901 benefit Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 238000007621 cluster analysis Methods 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012804 iterative process Methods 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 244000025254 Cannabis sativa Species 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013144 data compression Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000006837 decompression Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000003607 modifier Substances 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 238000004611 spectroscopical analysis Methods 0.000 description 1
- 238000009987 spinning Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
- G06V20/194—Terrestrial scenes using hyperspectral data, i.e. more or other wavelengths than RGB
Abstract
A method of clustering and reducing hyperspectral image data having a plurality of spatial pixels, and a plurality of spectral dimensions associated with each spatial pixel, includes computing an initial basis vector associated with the hyperspectral image data, unmixing the initial basis vector with the hyperspectral image data to generate an initial set of coefficients and an associated set of residual vectors, generating a set of clusters based on the initial set of coefficients, and iteratively computing one or more additional basis vectors and updating the set of clusters. The iterative computing includes calculating a subsequent basis vector based on a residual vector associated with a prior unmixing, unmixing the subsequent basis vector with a prior set of residual vectors to generate additional coefficients associated with each pixel, and iteratively computing cluster centers and content including an additional dimension associated with the subsequent basis vector.
Description
SYSTEM AND METHOD FOR REDUCED INCREMENTAL SPECTRAL CLUSTERING
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application is related to U.S. Patent Application Serial Number 13/488,255, entitled "System and Method for Rapid Cluster Analysis of Hyperspectral Images," filed June 4, 2012, U.S. Patent Application Serial Number 13/371,160, entitled "System and Method for Hyperspectral Image Compression," filed February 10, 2012, U.S. Patent Application Serial Number 13/085,883, entitled "Optimized Orthonormal System and Method for Reducing Dimensionality of Hyperspectral Images," filed April 13, 2011, U.S. Patent Application Serial No. 12/475,145, entitled "System and Method for Reducing Dimensionality of Hyperspectral Images," filed May 29, 2009, and U.S. Patent Application Serial No.
11/856,588, entitled "Hyperspectral Image Dimension Reduction System and Method," filed September, 17, 2007, the disclosures of each of which are hereby incorporated by reference in their entireties.
BACKGROUND
[0002] This disclosure relates to analysis of hyperspectral image data, and as described in greater detail below, relates in particular to creating dimensionality reduced hyperspectral image data, which may be based on an optimized set of basis vectors. While compression reduces the size of a data set, it typically results in a loss of access to information content. On the other hand, dimensionality reduction techniques provide compression with the ability to extract information from the data set in its reduced size. Thus, while all dimensionality reduction techniques provide compression, not all compression techniques allow for dimensionality reduction.
[0003] Hyperspectral sensors can collect image data across a multitude of spectral bands through a combination of technology associated with spectroscopy and remote imaging. Thus, such sensors can capture sufficient information to derive an approximation of the spectrum for each pixel in an image. In addition to having a color value, each pixel in the image additionally has a third dimension for a vector providing distinct information for the pixel over a large spectrum of wavelengths. This contiguous spectrum may be analyzed to
separate and evaluate differing wavelengths, which may permit finer resolution and greater perception of information contained in the image. From such data, hyperspectral imaging systems may be able to characterize targets, materials, and changes to an image, providing a detection granularity which may exceed the actual resolution of pixels in the image and a change identification capability that does not require pixel level registration, which may provide benefits in a wide array of practical applications.
[0004] Because each pixel carries information over a wide spectrum of wavelengths, the size of a hyperspectral data set may often quickly become unwieldy in terms of the size of data that is being recorded by the hyperspectral sensor. As an example, hyperspectral sensors are often located remotely on satellites or aircraft capable of imaging areas in excess of 500 km x 500 km per hour, which may result in the hyperspectral sensors generating anywhere from three to fifteen gigabits of data per second. Where the hyperspectral data needs to be processed in near real time, the large size of the data may introduce latency problems. In some cases, it may be desirable to transmit the data to a remote location for processing or other analysis, which again would make a reduced data size desirable. Additionally, it may be appreciated that large quantities of data may be difficult to analyze.
[0005] While lossy and/or lossless compression techniques may increase the transmission and processing rate for hyperspectral images, these techniques also suffer from various drawbacks. For example, while lossy compression methods may be fine for casual photographs or other human viewable images, wherein the data that is removed may be beyond the eye's ability to resolve, applying such lossy compression methods to a
hyperspectral data set may remove information that is valuable and desired for further computer or mathematical processing. Such removal of data may undermine the ability to characterize targets, materials, or changes to scenes that are captured in hyperspectral images. Lossless data compression would not remove such valuable information, since lossless algorithms produce a new data set that can subsequently be decompressed to extract the original data set. Although general purpose lossless compression algorithms can theoretically be used on any type of data, existing lossless compression algorithms typically cannot achieve significant compression on a different type data than that which the algorithms were designed to compress. Thus, existing lossless compression algorithms do not provide a suitable guaranteed compression factor for hyperspectral images, and in certain cases, the decompressed data set may even be larger than the original data set.
[0006] Dimensionality reduction techniques strike a balance between the loss of data resulting from lossy compression, and the increased processing requirements of lossless techniques. For example, the dimensionality reduction techniques may identify information that is of particular importance, and segregate it such that it is not reduced, while reducing the remaining information that is of less value. Thus, the use of dimensionality reduction on hyperspectral data sets allows for transformation of the hyperspectral image into a more compact form, with little to no loss of the most relevant information. At the same time, it is advantageous for dimensionality reduction techniques to facilitate rapid processing of a reduced hyperspectral image data set. In the case of hyperspectral imaging data, this generally means that the dimensionality reduced data may be exploited for target detection, anomaly detection, material identification, classification mapping, or so on. Typically for dimensionality reduction of hyperspectral images, a family of functions or a set of vectors are found whose arithmetic combination can represent all of the data in a three-dimensional (3D) data set. Hyperspectral image data is generally discrete, so at each X/Y location in a hyperspectral image the spectral data may form elements of a vector. Depending on the nature of these vectors, they may either be characterized as endmembers or basis vectors. While basis vectors span the data obtained from the image, and form a mathematical basis for the data, endmembers are pixels from an imaged scene (or extrapolations of pixels in the scene), that represent the spectra of a pure material found in the scene. In some cases, endmembers are derived such that they enclose or bound the data set (as in a hypervolume or a simplex).
[0007] It may be appreciated that some dimensionality reduction techniques such as those disclosed in the related applications incorporated by reference above, may compute geometric basis vectors. Dimensionality reduction may alternatively be achieved through other hyperspectral image processing mechanisms, including but not limited to Principal
Components Analysis, which computes "statistically derived' basis vectors that span a scene in an optimal mean-square sense.
[0008] This disclosure additionally relates to clustering of pixels of hyperspectral image data. Clustering is a process which finds pixels that are more similar to each other than to other groups of pixels. In particular, such clustering may be utilized to determine pixels containing like materials and outliers. It may be appreciated that when a scene is being imaged, different materials and different contrast within the scene may form a generally heterogeneous data set. Accordingly, clustering may break up the scene into more
homogeneous portions. Having identified homogenous regions of similar spectral properties, it may be easier to reject clutter. Additionally, such clusters may be utilized to create classification maps, which may be useful in characterizing the hyperspectral image. As one non-limiting example, a variety of clusters may be determined from the hyperspectral image data, including a cluster of pixels that represent grass, a cluster of pixels that represent water, a cluster of pixels that represent metal, and so on. It may be appreciated that clustering may also detect anomalies, such as determining those pixels that are outliers from the established clusters (or are identified as among the smallest clusters). Such clusters may further be useful in identifying pixels with similar spectral properties, which may be exploited in further analysis.
[0009] In a conventional implementation, clustering may include selecting an initial number of clusters followed by an initial assignment of every pixel to a cluster. In some cases, this assignment may be designated by a user. Tentative center coordinates for each cluster may be formed from the data or selected for each cluster. For example, the centers may be spaced uniformly to one another, or may be randomly distributed in the scene. Pixels may be assigned to the cluster whose center is nearest (e.g., the smallest distance to the center coordinates). The average coordinates of each cluster, including the added pixels, may then be computed to form the cluster center for the next iteration. In the next iteration, pixels would then be reassigned to the various clusters in an iterative process, with the iterations stopping when certain conditions are met. For example, in some implementations, a user- selected number of iterations may be performed. Alternatively, a stability condition may ultimately be reached, which signifies the end of the iterative process. For example, the cluster center may stop moving, or may move less than a certain distance. As another example, fewer than a certain percentage of pixels may change clusters from the past iteration. In still another example, each of the clusters may settle into a predetermined size or density range (e.g., the iterations end when clusters are not too small, too large, or
insufficiently dense). Constraints may be made on the configuration of an allowable cluster, such as by splitting clusters that are too small (redistributing the pixels from the small clusters into other appropriate cluster), splitting clusters that are too large into smaller clusters, or so on.
[0010] It may therefore be appreciated that conventional clustering, typically performed as a discrete process on full dimensioned hyperspectral image data, is generally a highly iterative and slow process. For example, many iterations of the computations, such as distance comparisons between pixels and each cluster, may be required to establish the clusters of pixels. Accordingly, among other things, it is advantageous to increase the speed at which stable clusters are identified. Speed may be increased by reducing the number of iterations and reducing the computations associated with each iteration. However, it is important that pixels are assigned to the correct clusters at the end of the clustering processing.
SUMMARY
[0011] According to an embodiment, a method of clustering and reducing hyperspectral image data having a plurality of spatial pixels, and a plurality of spectral dimensions associated with each spatial pixel, includes computing an initial basis vector associated with the hyperspectral image data. The method also includes unmixing the initial basis vector with the hyperspectral image data to generate an initial set of coefficients and an associated set of residual vectors. The method additionally includes generating a set of clusters based on the initial set of coefficients. The method further includes iteratively computing one or more additional basis vectors and updating the set of clusters. The iterative computing includes calculating a subsequent basis vector based on a residual vector associated with a prior unmixing. The iterative computing also includes unmixing the subsequent basis vector with a prior set of residual vectors to generate additional coefficients associated with each pixel. The iterative computing further includes iteratively computing cluster centers and content including an additional dimension associated with the subsequent basis vector.
[0012] According to another embodiment, a system configured to cluster and reduce hyperspectral image data having a plurality of spatial pixels, and a plurality of spectral dimensions associated with each spatial pixel, includes one or more processors. The one or
more processors are configured to compute an initial basis vector associated with the hyperspectral image data. The one or more processors are also configured to unmix the initial basis vector with the hyperspectral image data to generate an initial set of coefficients and an associated set of residual vectors. The one or more processors are additionally configured to generate a set of clusters based on the initial set of coefficients, and iteratively compute one or more additional basis vectors and update the set of clusters. The iterative computation includes calculating a subsequent basis vector based on a residual vector associated with a prior unmixing. The iterative computation further includes unmixing the subsequent basis vector with a prior set of residual vectors to generate additional coefficients associated with each pixel. The iterative computation further includes iteratively computing cluster centers and content including an additional dimension associated with the subsequent basis vector.
[0013] According to another embodiment, a method of clustering and reducing hyperspectral image data having a plurality of spatial pixels, and a plurality of spectral dimensions associated with each spatial pixel, includes computing an initial basis vector associated with the hyperspectral image data. The method also includes unmixing the initial basis vector with the hyperspectral image data to generate an initial set of coefficients and an associated set of residual vectors. The method additionally includes generating histograms of the initial set of coefficients to determine a number of pixels having a value within a given range of a value of the coefficient. The method further includes generating a set of clusters based on the histograms of the initial set of coefficients. The method furthermore includes iteratively computing one or more additional basis vectors and updating the set of clusters. The iteratively computing includes calculating a subsequent basis vector based on a residual vector associated with a prior unmixing. The iteratively computing also includes unmixing the subsequent basis vector with a prior set of residual vectors to generate additional coefficients associated with each pixel. The iteratively computing further includes iteratively generating histograms of the additional coefficients associated with the subsequent basis vector to identify new clusters.
[0014] Other features of this disclosure and the inventive concept described herein will be apparent to those skilled in the art based on the following drawings and detailed description.
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 illustrates a flowchart depicting an exemplary method for processing one or more hyperspectral images, according to an embodiment, based on a desired number of basis vectors;
[0016] Figure 2 illustrates a flowchart depicting an exemplary method for processing one or more hyperspectral images, according to an embodiment, based on stability of computed clusters;
[0017] Figure 3 illustrates a flowchart depicting an exemplary method for processing one or more hyperspectral images, according to an embodiment, based on histogrammed coefficients associated therewith;
[0018] Figure 4 schematically illustrates a division of clusters associated with sets of basis vectors and the histogrammed coefficient associated method of Figure 3;
[0019] Figure 5 illustrates an example of division of clusters utilizing the histogrammed coefficients from the method of Figure 3;
[0020] Figure 6 illustrates an exemplary hyperspectral imaging system having a hyperspectral image dimensionality reduction system associated therewith;
[0021] Figure 7 illustrates an exemplary hyperspectral image which may be processed according to one or more of the methods depicted herein; and
[0022] Figure 8 is a block diagram of a computer system in accordance with one or more embodiments.
DETAILED DESCRIPTION
[0023] Figure 1 illustrates an embodiment of a method of clustering and dimensionality reduction 100. As shown, in an embodiment, the method of clustering may include at 110 receiving hyperspectral image data (including, for example, a hyperspectral image). The hyperspectral image data received at 110 may be of any appropriate configuration, or may be received from any appropriate source. In an embodiment, the hyperspectral image data may be considered a hyperspectral image data cube, comprising both spatial and spectral dimensions. Specifically, the hyperspectral image data may be understood as generally containing two spatial dimensions, corresponding to the scene, and a third spectral dimension. In some embodiments, the hyperspectral image data may be characterized as vectors, having an X,Y spatial coordinate, and a set of values in the spectral dimension (e.g., associated with particular wavelength values). For example, in an embodiment, the hyperspectral image data received at 110 may be arranged to include a plurality of pixels associated with spatial locations therein, while each pixel includes spectral data associated with that spatial location. Additional dimensions in the hyperspectral image data are also possible in some embodiments, such that spectral vectors are recorded therein. In an embodiment, a system configured to perform the method of clustering and dimensionality reduction 100 may be the same system that generates the hyperspectral image data. As described in greater detail below, in some embodiments, receiving the hyperspectral image data cube at 110 may include utilizing the hyperspectral imaging system, which in some embodiments may include a remote imaging system (e.g., a satellite, an aerial surveillance system, or any other system that can capture hyperspectral images). In an embodiment, the hyperspectral imaging system may be configured to capture one or more hyperspectral images of a particular scene corresponding to a geographical area.
[0024] It may be appreciated that the hyperspectral imaging system may capture hyperspectral image data across the spectrum of wavelengths, thereby generating a data set corresponding to a hyperspectral image of scene. In some embodiments, a system configured to perform the method of clustering and dimensionality reduction 100 may be separate and discrete from that configured to capture or otherwise generate the hyperspectral image data cube. In such an embodiment, receiving the hyperspectral image data at 110 may include accessing stored hyperspectral image data, either stored locally to the system performing the
method of clustering and dimensionality reduction 100, or coupled to such a system (e.g., through a network).
[0025] As described in greater detail below, performing clustering of the hyperspectral image data received at 110 in an integrated manner with reducing the dimensionality thereof, may result in an increased speed of clustering the pixels thereof. In particular, the speed of clustering is generally highly dependent on the number of iterations and the computation of the cluster distances, as described above. Accordingly, reducing the time spent iterating and/or computing distances between pixels and the adjacent clusters is desirable. The method of clustering and dimensionality reduction 100 may be configured to compute basis vectors associated with the hyperspectral image data. Specifically, the basis vectors may be utilized in creating a dimensionally reduced configuration of the hyperspectral image data. As shown in the illustrated embodiment, after receiving the hyperspectral image data at 110, method 100 may continue at 120 by calculating a first basis vector associated with the hyperspectral image data. Calculating the first basis vector at 120 may be performed through any appropriate algorithm or computation, including but not limited to those disclosed in the related applications incorporated by reference above. For example, in an embodiment the first basis vector calculated at 120 (and/or subsequent basis vectors additionally calculated, as described below) may be derived without a priori knowledge of imaged scene, wherein the basis vectors may instead be derived to span background materials, statistically sparse targets, or other relevant objects within the scene. As an example, the basis vectors may be calculated from the hyperspectral image data in a manner that represents endmembers for pixel spectra associated with unique objects or other targets in scene imaged in the hyperspectral image data. In some embodiments, analysts or other users may alternatively or additionally select one or more pixel spectra to derive a specific basis vector.
[0026] After the first basis vector is calculated at 120, method 100 may continue by decomposing the first basis vector with the hyperspectral image data, to generate coefficients associated therewith. In the illustrated embodiment, such decomposing comprises, at 130, unmixing the basis vector against the hyperspectral image data. Unmixing, as used herein without further modifiers, is understood to generically refer to the broad concept of unconstrained unmixing. In an embodiment, the unmixing at 130 may comprise computing the dot product of the first basis vector with each (spatial) pixel of the hyperspectral image data, and a residual vector associated therewith. The computed dot product for each pixel is a
scalar value associated with the first basis vector, and may be understood as the first element of the dimensionality reduced data, an unmixing coefficient of the dimensionally reduced data. Both the computed dot product (the scalar value) and the residual vector may be stored, and may be accessed during the remainder of the performance of method 100, or during subsequent processing. In an embodiment, the unmixing at 130 may include identifying outliers or anomalies, which might not be dimensionally reduced, but may instead be set aside so as to be retained as unreduced pixels suitable for subsequent analysis. For example, in an embodiment the outliers may be determined incrementally (e.g., while unmixing a first basis vector with the hyperspectral image data might not indicate any of the pixels as outliers, unmixing a second basis vector with the hyperspectral image data may indicate that certain pixels are anomalous). As such, in an embodiment, outliers may be identified on a per- coefficient basis. In an embodiment, outliers may be based on a lack of similarity to other clusters, as described in greater detail below, since pixels that differ from all clusters in some basis vector would not get merged into those clusters based on the values resulting from other basis vectors.
[0027] Method 100 may continue at 140 by establishing initial cluster assignments based on the coefficients obtained through the unmixing at 130. In an embodiment, the initial clustering at 140 may include assigning pixels to various clusters based on the scalar coefficient associated therewith. For example, in an embodiment, pixels having low scalar values may be assigned to a first cluster, pixels having intermediate scalar values may be assigned to a second cluster, and pixels having high scalar values may be assigned to a third cluster. In an embodiment, the coefficients may range from -1 to 1. In an embodiment, the coefficient values may generally be divided into equally spaced centers (e.g., if there are 7 centers selected, they may be located at -3/4, -1/2, -1/4, 0, ¼, ½, and ¾). In an embodiment, the number of centers (e.g., number of initial clusters) may be a user-selected value. It may be appreciated that any appropriate number of initial centers are possible, and may be user- selected as an input.
[0028] The division of pixels into initial cluster assignments may vary across
embodiments. For example, in some embodiments the scalar value determination of cluster assignment may depend on the number of clusters. The number of clusters may be based on a user-selected value, as indicated above. In another embodiment, the number of clusters may depend on the scalar values identified in the unmixing at 130. For example, where a variance
between the scalar values indicates a grouping between certain pixels (e.g., where certain sets of scalar values are closer together than other sets of scalar values) those groupings may be utilized to establish not only the initial cluster assignments, but the number of initial clusters. It may be appreciated that in some embodiments, the number of clusters may grow (e.g., a cluster may be split into multiple clusters), or the number of clusters may shrink (e.g., multiple clusters may be merged) as method 100 proceeds. In some embodiments where outliers or anomalies are identified during the unmixing, those anomalous pixels may be treated as separate clusters. In other such embodiments, the group of anomalous pixels may be treated as an "anomaly" cluster. In still other embodiments, the anomalous pixels may be excluded from being clustered.
[0029] After the initial clustering at 140 is complete, method 100 may continue at 150 by calculating a subsequent basis vector. It may be appreciated that the subsequent basis vector may be computed by any appropriate process, including but not limited to that disclosed in U.S. Patent Application Serial No. 13/085,883, incorporated by reference above. For example, the subsequent basis vector may be computed by normalizing the residual vector of those computed at 130 having a largest residual magnitude. It may be appreciated that the normalized residual vector may be orthogonal to the first basis vector calculated at 120, and with its normalization, may be an orthonormal vector. The orthonormal vector computed at 150 may thus be utilized as the subsequent basis vector. It may be appreciated that constraining each basis vector to be orthogonal to the previous basis vector has substantial advantages in the efficiency of processing, since the unmixing can proceed one basis vector at a time without consideration of correlations in the basis vectors which may modify previous calculated basis vector coefficients or clustering.
[0030] Method 100 may then continue at 160 by unmixing the subsequent basis vector computed at 150 with the residual vector at each pixel. As indicated above, such unmixing may comprise computing the dot product between the subsequent basis vector and the residual vector at each pixel. The unmixing at 160 may therefore obtain a subsequent coefficient associated with each pixel, which again may be understood to be a scalar value associated with each pixel.
[0031] Having computed the subsequent coefficient for each pixel, method 100 may continue at 170 by computing new cluster centers based on the subsequent scalar values. Such further computation of the cluster centers may be considered as refining the clusters. In an embodiment, the new cluster centers may be determined by computing the average values of all those pixels in each cluster. The average then becomes the new cluster center containing one additional dimension than the previous cluster center. Typically, the initial calculation of cluster centers, using the previous cluster assignments, will be the same in all previous orthonormal basis vectors, with an additional dimension added to the vector expressing the cluster center. As all pixel distances are recalculated with an additional dimension, however, some pixel assignments may change, resulting in the cluster centers changing. Additionally, because the new cluster is based on additional information (e.g., dimensions), the new cluster center will be more accurate than the prior cluster center.
[0032] Having computed the clusters based on the new cluster centers, the assignment of pixels to each cluster may be updated at 180. In an embodiment, the distance between each pixel and each new cluster center is then calculated, including the added dimension. In some embodiments the distance may be computed as the Euclidean distance (e.g., the geometric distance), or may be computed as the Mahalanobis distance (e.g., the distance being weighted by a spectral co variance of the cluster). If a pixel is closer to the center of a different cluster, it may be reassigned to that cluster. It may be appreciated that the "distance" computation is not necessarily a physical (e.g., spatial) distance, but is instead a hyperspectral distance that may be based on the endpoints of the spectral vector associated with each pixel.
[0033] Having updated the pixel assignments into each cluster at 180, method 100 may continue at 190 by determining whether to iterate cluster assignments. If so, method 100 may proceed by looping back and recalculating the cluster centers at 170 for the new pixel assignments and recalculating the pixel distances and assignments at 180 for the updated cluster centers. In some embodiments, the iteration determination at 190 may end (i.e., by determining not to iterate cluster assignments) after a specified number of iterations, or when cluster changes are sufficiently small (which, for example, may be based on a threshold value, comparing the movement of the cluster center to prior movements of the cluster center, or by any other appropriate determinative factor).
[0034] By making clustering decisions incrementally, based initially on a small number of coefficients, and updating the clusters as additional basis vectors are computed and unmixed to generate additional coefficients, it may be appreciated that the calculations for the clustering may be simpler than would be found when attempting to cluster based on a complete hyperspectral data set. In particular, it may be appreciated that such incremental computation of the basis vectors and coefficients and contemporaneous clustering may be more rapid than clustering based on either the original pixels of the hyperspectral image data, or even clustering based on a full set of dimensionally reduced hyperspectral image data, such as is discussed in U.S. Patent Application Serial No. 13/488,255, incorporated by reference above. Specifically, the initial iterations, which utilize a small number of coefficients associated with each pixel, may be computed very quickly. As additional basis vectors and coefficients are computed (in essence adding elements to the dimensionality reduced data), the movement of the clusters would be more stable, further facilitating less intense computations, and resulting in a more efficient process overall.
[0035] It may be appreciated that the processes of calculating subsequent basis vectors and reclustering based thereon in method 100 may continue until a completion condition is met. As shown in the illustrated embodiment, where there is a fixed number of basis vectors, method 100 may continue by determining at 200 if the current basis vector that has been calculated and utilized in the clustering is the last basis vector. If not, then method 100 may return to 150, calculating still another subsequent basis vector. Such looping of the calculation of basis vectors and reclustering may continue until it is determined at 200 that there are no additional basis vectors to compute.
[0036] When it is determined at 200 that the current basis vector is the last basis vector, then method 100 may end at 210. In an embodiment, such termination at 210 may comprise outputting the set of coefficients (arranged in an array or other usable format), which may be considered dimensionally reduced hyperspectral image data. As noted above, in some embodiments outliers, such as statistical anomalies from the scene, may be determined during the unmixing, and may be output separately from the dimensionally reduced hyperspectral image data. Additionally, clustering data associated with the hyperspectral image data (e.g., a listing of each cluster, and the pixels associated therewith) may also be output at 210. By being "output," the dimensionally reduced hyperspectral image data, the clustering data, outliers, and/or other associated data may be displayed, stored, saved (e.g., in a memory
device or in persistent storage), or otherwise made accessible for future computations, analysis, or consideration.
[0037] In an embodiment, the outputting at 210 may comprise outputting a classification map. For example, where the clusters represent different materials, the classification of each cluster may correspond to identification of the materials in the scene. As a specific example, those pixels representing a first material (e.g., water) may be highlighted as such in the classification map, while those pixels representing a second material (e.g., earth) may be highlighted differently from the first material. As noted, in an embodiment, anomalies may also be output at 210. In an embodiment, outputting the anomalies at 210 may include declaring certain pixels as anomalous, which may include highlighting them in the classification map as anomalous pixels. While in some embodiments all anomalous pixels may be highlighted as such with the same identifier indicating that the pixel is anomalous, on other embodiments where the anomalous pixels are distinct from one another, declaring the anomalous pixels may include providing a separate identifier for each anomalous pixel. For example, each distinct anomalous pixel or anomalous set of pixels may be treated as a separate cluster.
[0038] In some embodiments, a stability condition associated with the clusters may be configured to determine that the method of dimensionality reduction and clustering is complete. For example, Figure 2 illustrates a method 220 of dimensionality reduction and clustering. In some embodiments, portions of the method 220 may be generally similar to embodiments of those described above in method 100. As such, Figure 2 identifies similar processes with the same identifier as utilized in Figure 1. Specifically, method 220 may begin at 110 by receiving the hyperspectral image data, as described above. Method 220 may then continue at 120 by calculating a first basis vector, and proceed at 130 by unmixing the first basis vector with the pixels of the hyperspectral image data. Method 220 may then continue at 140 by clustering based on the coefficients derived from the unmixing.
[0039] Having established the initial basis vector, coefficients, and clusters, method 220 may continue at 150 by calculating a subsequent basis vector, and proceed further at 160 by unmixing the subsequent basis vector. Method 220 may then also proceed at 170 by computing new cluster centers based on the results of the unmixing at 160. Having computed the new cluster centers at 170, method 220 may continue at 180 by updating pixel distances
and assignments as described above. Method 220 may then proceed to 190 with determining whether to iterate cluster assignments, as described above. If so, iterating cluster assignments at 190 may include returning to 170 and recomputing the new cluster centers at 170, before continue at 180 by updating pixel distances and assignments.
[0040] As Figure 2 illustrates, if it is determined at 190 that cluster assignments are not to be updated, then in an embodiment method 220 may continue at 230 by determining if the clusters have been stable since the addition of the most recent basis vector.. In an
embodiment, the prior iteration may be based on the clustering at 140, while in other embodiments, the prior iteration may be from a prior computation of a basis vector at 150, as described below. In some embodiments, a minimum number of computations may be required before the determining of cluster stability at 230 is performed.
[0041] In an embodiment, determining that the clusters are stable at 230 may comprise determining if the cluster center did not move, or moves less than a certain (e.g., user defined or pre-determined) amount, from one iteration to the next. As another example, if less than a certain (e.g., user defined or pre-determined) number of pixels change clusters, then the clusters may also be considered stable. Other cluster rules may be applied to determine when the iterative basis vector and coefficient computation, and contemporaneous clustering may conclude. It may be appreciated that in embodiments where the number of pixels, or the size of cluster center movement, are utilized as a stability condition, and are user defined, the defined threshold value may be input by any appropriate mechanism, including through any suitable user interface. In some embodiments utilizing pixels changing clusters as a predetermined stability condition, the stability condition for pixels changing clusters may be, for example, set as less than a number of the pixels (e.g., less than 50 pixels changing clusters), or may be set as less than a fraction or percent of the overall pixels (e.g., less than 5% of the pixels changing clusters). Likewise, in embodiments where movement of the cluster center is a stability condition, the stability condition may be determined as a distance in hyperspectral dimensions (e.g., movement of the cluster center is less than a specified distance or angular change from the prior iteration), or may be as a fraction or percentage (e.g., movement of the cluster center from the prior iteration is less than 5%of the distance to the nearest cluster).
[0042] If the clusters are determined to not be stable at 230 (e.g., the center of the cluster and/or the pixels associated therewith change by some determined amount during method 220 from one iteration to the next), method 220 may continue at 240 by determining whether additional basis vectors are to be computed. In some embodiments, such determining at 240 may be analogous the determining at 200 of method 100. As such, if it is determined at 240 that the current basis vector is not the last basis vector, method 220 may return to 150, wherein a subsequent basis vector is calculated. In an embodiment, the determining at 240 may generally assume that if the clusters were not stable at 230, an additional basis vector should be calculated, and the iteration may continue by returning to 150. In some such embodiments, determining whether the current basis vector is the last basis vector at 240 may comprise determining whether an additional basis vector can be computed. For example, the determining at 240 may include computing whether calculating an additional basis vector and generating coefficients associated therewith would be of any greater dimensionality reduction benefit than utilizing the unreduced hyperspectral image data. Such computation may include calculating an error associated with the dimensionally reduced data (from the coefficients) as compared to the hyperspectral image data, and determining if the error is greater or less than a threshold value (e.g., one that is user determined). Such computation may also or alternatively include calculating a data size associated with the dimensionally reduced as compared to that of the hyperspectral image data, and determining if the data size is greater or less than a threshold value (e.g., pre-set, user determined, or so on).
[0043] If it is determined at 240 that the clusters have been stable since the last iteration of computing the subsequent basis vector, method 220 may continue at 250 by either stopping (which may be generally similar to terminating at 260, as described below), or continuing by calculating new basis vectors specific to each cluster. If continuing at 250 means that method 220 stops, then it may be determined that additional basis vectors are not needed, and stopping at 250 may include informing the user that additional basis vectors are unnecessary. Where it is determined that new basis vectors are to be calculated specific to each cluster, continuing at 250 may include treating the pixels in each stable cluster as a separate dataset, and adding additional basis vectors for each cluster. In an embodiment, continuing at 250 may include testing whether the clusters should be split into additional classes when adding the additional basis vectors. Specifically, when additional dimensions of
information are added to each cluster, pixels in a given cluster may have different values in those dimensions, thus splitting the cluster.
[0044] After the last basis vector is determined to have been calculated at 240, or after continuing at 250 is complete (either by stopping or by completing calculating the new basis vectors specific to each cluster), method 220 may terminate at 260. Similarly to terminating at 210, terminating at 260 may comprise outputting the set of coefficients as dimensionally reduced hyperspectral imaging data, outputting the set of basis vectors associated therewith, outputting outliers/anomalies, and/or outputting the clustering data. Other data computed or manipulated in method 220 may also be output when terminating at 260.
[0045] In some embodiments, the clustering and dimensionality reduction may be configured to utilize histogramming analysis. Specifically Figure 3 illustrates a method 270 of clustering and dimensionality reduction. Method 270 includes at 280 receiving hyperspectral image data. In an embodiment, receiving the hyperspectral image data at 280 may be similar to receiving the hyperspectral image data at 110 as described above with regard to method 100 and method 220. Having received the hyperspectral image data at 280, method 270 may continue to 290, by calculating a first basis vector associated with the hyperspectral image data, and further proceed to 300 by unmixing the first basis vector with the hyperspectral image data to generate the set of coefficients associated with the hyperspectral image data. It may be appreciated that calculating the first basis vector at 290 and unmixing the first basis vector at 300 may generally be similar to calculating the first basis vector at 120 and unmixing the first basis vector at 130 as described above with regard to method 100 and method 200.
[0046] Once the coefficients associated with the first basis vector are generated at 300, method 270 may continue at 310 by histogramming the coefficients. As illustrated in Figures 4 and 5, histogramming the coefficients at 310 may comprise plotting the coefficients in a manner that allows identification of clusters therein. For example, in the illustrated embodiment, the x-axis is the value of the first basis vector coefficient, and the y-axis is the number of pixels within a given range of x (e.g., the number of pixels with first basis vector coefficients between 0.9 and 1.1). In an embodiment, the plot may be generated by creating the bins for the histogram and counting the number of pixels with values within each bin. It may be appreciated in Figure 5 that in some embodiments, local minimums along the x-axis
allow divisions between groups of coefficients to establish cluster 1, cluster 2, and cluster 3 (CI, C2, and C3 in Figure 4 and Figure 5).
[0047] Returning to Figure 3, once the coefficients from the first basis vector are histogrammed at 310, method 270 may continue at 320 by splitting the pixels associated with the coefficients into clusters separated by local minimums of the histogram, and calculating a center of the cluster based on the pixels being in the cluster via the histogram. Method 270 may then continue by calculating a subsequent basis vector at 330. In an embodiment, calculating the subsequent basis vector at 330 may be similar to calculating the subsequent basis vector at 150 of method 100 or method 220. Once the subsequent basis vector is calculated at 330, it may be unmixed with the hyperspectral image data at 340. Unmixing the subsequent basis vector at 340 may generally be similar to unmixing the subsequent basis vector at 160, as described above with reference to method 100 or method 220.
[0048] As shown in Figure 3, method 270 may continue at 350 by histogramming the coefficients of the subsequent basis vector generated at 330 for each cluster generated at 320. Specifically, the histogramming at 350 may comprise selecting only the subset of pixels that form the cluster computed at 320, and plotting the number of pixels in each second basis vector bin versus the center value of the bin associated with the second basis vector computed at 330. In Figure 5, it may be appreciated that the data associated with the first basis vector ("BV1") is understood to be extending out of the Figure. As such, by viewing the histogram along the axis established by the subsequent basis vector (e.g., "BV2"), the distribution of coefficients from the subsequent basis vector may be seen and analyzed for each cluster previously formed from the previous basis vectors. Again, local minimums may present themselves throughout the histogrammed distribution, lending themselves to additional clusters being identified at 360. In the examples of Figures 4 and 5, where there are three clusters identified with the first basis vector (i.e., CI, C2, and C3) and two additional clusters split from the first cluster when the second basis vector is considered, the subsequent clusters split from CI may be sequentially C4 and C5. Once the additional clusters are identified at 360, a center of the new clusters may be calculated. It may be appreciated that the number of pixels associated with CI will decrease when C4 and C5 are identified. As such, CI may be understood as splitting into a "new" CI, as well as C2 and C3 as illustrated herein. As shown in Figure 3, the histogrammed cluster analysis may continue at 370 by determining if the initial clusters determined based on the first basis vector have been split. If not, method 270
may return to histogramming the coefficients of the subsequent basis vector at 350. For example, in the examples of Figures 4 and 5, during subsequent iterations, clusters C6 and C7 may be identified based on clusters C2 and C3 respectively.
[0049] Once all clusters have been identified, it may be determined at 370 that there are no additional clusters associated with the current basis vector iteration. Method 270 may then continue at 380 by determining if there are additional basis vectors to identify. In an embodiment, the conditions to determine if there are additional basis vectors may be similar to identifying if it is the last basis vector at 200 or 240 in methods 100 and 220 above. For example, the number of basis vectors may be fixed, or a stability condition may be tested for. If it is determined at 380 that there are additional basis vectors to calculate, method 270 may return to 330, to calculate the subsequent basis vector. If there are no more basis vectors to calculate, however, method 270 may terminate at 390. It may be appreciated that terminating at 390 may be generally similar to terminating at 210 or 260, as discussed above as relating to methods 100 or 220 respectively.
[0050] It may be appreciated that in some embodiments, performing the clustering and dimensionality reduction may result in dimensionally reduced data that may be utilized for further analysis without need for decompression. Specifically, the dimensionally reduced data may comprise coefficients that indicate the greatest contribution of the pixel to the scene. Although the dimensionalities of the hyperspectral image data dimensionally reduced hyperspectral image data may vary across embodiments, in one non-limiting embodiment, the hyperspectral image data may include approximately 256 spectral dimensions, while the reduced dimensionality data may include approximately 15 spectral dimensions. It may be appreciated that in other embodiments, while the dimensionally reduced configuration retains the same number of spatial pixels as the original hyperspectral image data, the number of elements in the spectral dimension may be reduced. Additionally, in some embodiments, the dimensionally reduced data may also be treated as a set of vectors (e.g., dimensionally reduced vectors), having an X,Y spatial coordinate, and a set of values associated therewith, indicating the reduced dimensionality data. In various embodiments the hyperspectral image data cube may be received from any appropriate source. Additionally, in some embodiments, a system configured to perform the methods of clustering and dimensionality reduction may be the same system that generates the hyperspectral image data. In other embodiments, the
system that generates the hyperspectral image data may be linked to the system that performs the methods of dimensionality reduction and clustering through any appropriate mechanism.
[0051] Figure 6 illustrates an exemplary image dimension reduction system ("IDRS") 410 incorporated as part of a hyperspectral image system 420 provided by a satellite 430. The satellite 430 is tasked to image a geographical area 440, and in so doing receives light 450 from a single ground-resolution cell. To generalize the basic elements provided by a hyperspectral image system 420 and the general method of operation, the light 450 is received by scan mirrors 460 and or other optics that direct the light through at least one dispersing element 470. The dispersing element separates wavelengths and provides them to imaging optics 480 which in turn focus the wavelengths upon an array of detectors 490 arranged to capture data information across a spectrum of wavelengths. The hyperspectral data is then processed by IDRS 410 for transmission 500 to a ground station 510.
[0052] While the hyperspectral image data cube created by the hyperspectral image system 420 may vary across embodiments, Figure 7 illustrates an embodiment of a hyperspectral image 520 (i.e., a hyperspectral image data cube) including constituent images (e.g., images 520i~n) acquired simultaneously in many different adjacent wavelength bands. As indicated, the image is comprised of pixels arranged in an X-Y coordinate system. Under appropriate circumstances, alternative coordinate systems may be employed, however X-Y is represented herein for ease of illustration and discussion, and not by way of limitation. Each pixel 530 of the hyperspectral image 520 has a spectral vector 540. The spectral vector 540 contains at least some number of spectral measurements of the energy upwelling from that pixel. As indicated in an enlarged section 550, of row X' the spectral vector 540A of the initial pixel 530A may well be different from the spectral vectors 540B~540n for pixels 530B~530n.
[0053] Although various embodiments of the methods of clustering and dimensionality reduction described herein may be implemented on any appropriate system or hardware, in some embodiments, the methods of clustering and dimensionality reduction may be implemented on a computer system, which may generally include typical computer components such as one or more processors, memory modules, storage devices, input and output devices, and so on. In an embodiment, the methods of clustering and dimensionality reduction may be maintained in an active memory of the computer system to enhance speed
and efficiency, and may further be coupled to a computer network and utilize distributed resources associated with the computer network. In various embodiments, a system configured to implement the methods of clustering and dimensionality reduction may include one or more interfaces, one or more spectrum readers, and one or more modules that may perform establishing the basis vector set, decomposing the hyperspectral image, and evaluating the hyperspectral image. In some embodiments, the one or more interfaces may be configured to receive data corresponding to one or more hyperspectral images, one or more basis vectors provided by a user, an indication as to whether dimensionality reduction is to be performed as a lossy or lossless operation, tolerance levels for the amount of lost data in the dimensionality reduction, and/or other information relating to the processing and/or analyzing of hyperspectral images. In an embodiment, the one or more interfaces may be arranged to receive information directly from the user via an input device associated with the system, or directly from a component of the system, including, for example, a hyperspectral imager.
[0054] According to an embodiment, implementations of the various systems and methods for clustering and reducing dimensionality of hyperspectral images described herein may be made in hardware, firmware, software, or various combinations thereof. For example, the systems and methods for clustering and reducing dimensionality of hyperspectral images may be implemented as computer executable instructions stored on a non-transitory machine readable medium, which may be read and executed using one or more physically separate or communicatively coupled computer systems or other processing devices. The machine readable medium may include various mechanisms for storing and/or transmitting
information in a manner readable by the computer systems, the processing devices, or other machines. For example, a machine readable storage medium may include read only memory, random access memory, magnetic disk storage media, optical storage media, flash memory devices, hard drives, cache, removable media, network storage, the internet, the "cloud," another system, or any other appropriate device and/or other media for storing information, and a machine readable transmission media may include signals such as carrier waves, infrared signals, digital signals, and other media for transmitting information. It may be appreciated that receiving or accessing data associated with the methods described herein may be performed at any time and/or in any order. For example, data may be received from storage and loaded into active memory for use as needed, or may be received all at once (e.g., simultaneously, or sequentially at one time). Additionally, although the above disclosure may
describe methods, firmware, software, routines, or instructions in terms of specific exemplary aspects and implementations and performing certain actions, it will be apparent that such descriptions are merely for the sake of convenience and that such actions in fact result from the computer systems, the processing devices, processors, controllers, or other devices or machines executing the firmware, software, routines, or instructions.
[0055] Figure 8 illustrates a high level block diagram of an exemplary computer system 560 which may be used to perform the methods of clustering and dimensionality reduction, and/or similar embodiments. It may be appreciated that in some embodiments, the IDRS 410 may include some or all of the computer system 560. In some embodiments, the computer system 560 may be linked to or otherwise associated with the ground station 510. In an embodiment the computer system 560 has a case 570, enclosing a main board 580. The main board has a system bus 590, connection ports 600, a processing unit, such as Central Processing Unit (CPU) 610, and a data storage device, such as main memory 620, storage drive 630, and optical drive 640. Each of main memory 620, storage drive 630, and optical drive 640 may be of any appropriate construction or configuration. For example, in some embodiments storage drive 630 may comprise a spinning hard disk drive, or may comprise a solid-state drive. Additionally, optical drive 640 may comprise a CD drive, a DVD drive, a Blu-ray drive, or any other appropriate optical medium.
[0056] Memory bus 650 couples main memory 620 to CPU 610. A system bus 590 couples storage drive 630, optical drive 640, and connection ports 600 to CPU 610. Multiple input devices may be provided, such as for example a mouse 660 and keyboard 670. Multiple output devices may also be provided, such as for example a video monitor 680 and a printer (not shown). It may be appreciated that the input devices and output devices may
alternatively be local to the case 570 and the computer system 560, or may be located remotely (e.g., interfacing with the computer system 560 through a network or other remote connection).
[0057] Computer system 560 may be a commercially available system, or may be proprietary design. In some embodiments, the computer system 560 may be a desktop workstation unit, and may be provided by any appropriate computer system provider. In some embodiments, computer system 560 comprise a networked computer system, wherein memory storage components such as storage drive 630, additional CPUs 610 and output devices such as printers are provided by physically separate computer systems commonly tied together in the network. Those skilled in the art will understand and appreciate the physical composition of components and component interconnections comprising computer system 560, and select a computer system 560 suitable for performing the methods disclosed herein.
[0058] When computer system 560 is activated, preferably an operating system 690 will load into main memory 620 as part of the boot sequence, and ready the computer system 560 for operation. At the simplest level, and in the most general sense, the tasks of an operating system fall into specific categories—process management, device management (including application and user interface management) and memory management.
[0059] In such a computer system 560, the CPU 610 is operable to perform one or more embodiments of the methods described above. Those skilled in the art will understand that a computer-readable medium 700 on which is a computer program 710 for performing the methods (e.g., the methods of clustering and dimensionality reduction 100, 220, and 270) disclosed herein may be provided to the computer system 560. The form of the medium 700 and language of the program 710 are understood to be appropriate for computer system 560. Utilizing the memory stores, such as one or more storage drives 630 and main system memory 620, the operable CPU 610 will read the instructions provided by the computer program 710 and operate to perform the methods of clustering and dimensionality reduction as described above.
[0060] It may be appreciated that aspects and implementations may be described in the above disclosure as including particular features, structures, or characteristics, but it will be apparent that every aspect or implementation may or may not necessarily include the particular features, structures, or characteristics. Further, where particular features, structures, or characteristics have been described in connection with a specific aspect or implementation, it will be understood that such features, structures, or characteristics may be included with other aspects or implementations, whether or not explicitly described. Thus, various changes
and modifications may be made to the preceding disclosure without departing from the scope or spirit of the inventive concept, and the specification and drawings should therefore be regarded as exemplary only, with the scope of the invention determined solely by the appended claims.
Claims
1. A method of clustering and reducing hyperspectral image data having a plurality of spatial pixels, and a plurality of spectral dimensions associated with each spatial pixel, the method comprising:
computing an initial basis vector associated with the hyperspectral image data;
unmixing the initial basis vector with the hyperspectral image data to generate an initial set of coefficients and an associated set of residual vectors;
generating a set of clusters based on the initial set of coefficients; and
iteratively computing one or more additional basis vectors and updating the set of clusters by:
calculating a subsequent basis vector based on a residual vector associated with a prior unmixing;
unmixing the subsequent basis vector with a prior set of residual vectors to generate additional coefficients associated with each pixel; and
iteratively computing cluster centers and content including an additional dimension associated with the subsequent basis vector.
2. The method of claim 1 , wherein said unmixing the initial basis vector with the hyperspectral image data comprises computing a dot product of the initial basis vector with each spatial pixel of the hyperspectral image data, and computing a residual vector associated with each dot product.
3. The method of claim 1 further comprising outputting dimensionally reduced data comprising one or more of the initial set of coefficients and the additional coefficients associated with each pixel.
4. The method of claim 1, further comprising identifying and setting aside outliers identified while iteratively computing one or more additional basis vectors and updating the set of clusters.
5. The method of claim 4, wherein the outliers are identified by dissimilarity from each of the set of clusters.
6. The method of claim 1, wherein the set of clusters based on the initial set of coefficients are equally spaced across the hyperspectral image data.
7. The method of claim 6, wherein there are a user-selected number of the initial clusters.
8. The method of claim 1, wherein iteratively recomputing cluster centers comprises splitting a prior cluster into multiple new clusters.
9. The method of claim 1, wherein iteratively recomputing cluster centers comprises merging two or more prior clusters into a new cluster.
10. The method of claim 1, wherein the initial basis and the subsequent basis vector are orthonormal to each other.
11. The method of claiml , wherein the subsequent basis vector is computed by normalizing the residual vector associated with a prior unmixing.
12. The method of claim 1, wherein iteratively recomputing cluster centers and content comprises computing average values of the pixels currently assigned to each cluster.
13. The method of claim 1, wherein iteratively recomputing cluster centers and content comprises computing a distance between each pixel of the hyperspectral image data and each cluster center including an additional dimension, and assigning each pixel to a cluster having a closest distance
14. The method of claim 13, wherein the distance is computed as the Euclidean distance or the Mahalanobis distance.
15. The method of claim 1, wherein iterating cluster assignments and content comprises recomputing cluster centers based on the pixels reassigned to each cluster following the subsequent unmixing.
16. The method of claim 1, wherein said iterating cluster assignments continues for a specified number of iterations.
17. The method of claim 1, wherein changes in the clusters determines when to terminate said iteratively recomputing cluster centers and content.
18. The method of claim 1, further comprising outputting clustered data associated with the hyperspectral image data
19. The method of claim 18, wherein the clustering data comprises a classification map.
20. The method of claim 1, further comprising determining whether clusters are stable since a prior iteration of iteratively computing one or more additional basis vectors.
21. The method of claim 20, further comprising, when clusters are stable since the prior iteration, terminating without computing a subsequent iteration of iteratively computing one or more additional basis vectors.
22. The method of claim 20, further comprising, when clusters are stable since the prior iteration, computing new basis vectors specific to each cluster.
23. A system configured to cluster and reduce hyperspectral image data having a plurality of spatial pixels, and a plurality of spectral dimensions associated with each spatial pixel, the system comprising:
one or more processors configured to:
compute an initial basis vector associated with the hyperspectral image data; unmix the initial basis vector with the hyperspectral image data to generate an initial set of coefficients and an associated set of residual vectors;
generate a set of clusters based on the initial set of coefficients; and iteratively compute one or more additional basis vectors and update the set of clusters by:
calculating a subsequent basis vector based on a residual vector associated with a prior unmixing;
unmixing the subsequent basis vector with a prior set of residual vectors to generate additional coefficients associated with each pixel; and
iteratively computing cluster centers and content including an additional dimension associated with the subsequent basis vector.
24. A method of clustering and reducing hyperspectral image data having a plurality of spatial pixels, and a plurality of spectral dimensions associated with each spatial pixel, the method comprising:
computing an initial basis vector associated with the hyperspectral image data;
unmixing the initial basis vector with the hyperspectral image data to generate an initial set of coefficients and an associated set of residual vectors;
generating histograms of the initial set of coefficients to determine a number of pixels having a value within a given range of a value of the coefficient
generating a set of clusters based on the histograms of the initial set of coefficients; and
iteratively computing one or more additional basis vectors and updating the set of clusters by:
calculating a subsequent basis vector based on a residual vector associated with a prior unmixing;
unmixing the subsequent basis vector with a prior set of residual vectors to generate additional coefficients associated with each pixel; and
iteratively generating histograms of the additional coefficients associated with the subsequent basis vector to identify new clusters.
25. The method of claim 24 wherein generating the set of clusters comprises identifying local minimums in the histograms of the initial set of coefficients, and calculating a center of a cluster based on pixels positioned between the local minimums.
26. The method of claim 24 wherein iteratively generating histograms of the additional coefficients comprises identifying new local minimums in histograms of the additional coefficients, and splitting the cluster at the local minimums.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP13732757.3A EP2880596B1 (en) | 2012-08-03 | 2013-06-12 | System and method for reduced incremental spectral clustering |
IL237045A IL237045B (en) | 2012-08-03 | 2015-02-02 | System and method for reduced incremental spectral clustering |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/566,914 US8660360B1 (en) | 2012-08-03 | 2012-08-03 | System and method for reduced incremental spectral clustering |
US13/566,914 | 2012-08-03 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2014021996A1 true WO2014021996A1 (en) | 2014-02-06 |
Family
ID=48703899
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2013/045422 WO2014021996A1 (en) | 2012-08-03 | 2013-06-12 | System and method for reduced incremental spectral clustering |
Country Status (4)
Country | Link |
---|---|
US (1) | US8660360B1 (en) |
EP (1) | EP2880596B1 (en) |
IL (1) | IL237045B (en) |
WO (1) | WO2014021996A1 (en) |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110363712A (en) * | 2019-06-14 | 2019-10-22 | 江苏理工学院 | A kind of high spectrum image solution mixing method of sparse dual constraint |
CN115834895A (en) * | 2023-02-22 | 2023-03-21 | 创银科技(南通)有限公司 | Data efficient compression storage method for unmanned aerial vehicle |
Families Citing this family (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8675989B2 (en) * | 2011-04-13 | 2014-03-18 | Raytheon Company | Optimized orthonormal system and method for reducing dimensionality of hyperspectral images |
JP2015002462A (en) * | 2013-06-17 | 2015-01-05 | ソニー株式会社 | Image compression circuit, image compression method, and transmission system |
US9945721B2 (en) * | 2013-06-20 | 2018-04-17 | The University Of North Carolina At Charlotte | Selective wavelength imaging systems and methods |
JP6191505B2 (en) | 2014-02-28 | 2017-09-06 | ブラザー工業株式会社 | Image processing apparatus and image processing program |
KR101623826B1 (en) * | 2014-12-10 | 2016-05-24 | 주식회사 아이디스 | Surveillance camera with heat map |
US10089319B2 (en) * | 2015-02-20 | 2018-10-02 | International Business Machines Corporation | Policy-based, multi-scheme data reduction for computer memory |
US10337294B2 (en) * | 2015-10-01 | 2019-07-02 | Baker Hughes, A Ge Company, Llc | Reservoir permeability upscaling |
JP7057913B2 (en) * | 2016-06-09 | 2022-04-21 | 株式会社島津製作所 | Big data analysis method and mass spectrometry system using the analysis method |
CN108846259B (en) * | 2018-04-26 | 2020-10-23 | 河南师范大学 | Gene classification method and system based on clustering and random forest algorithm |
CN109374533B (en) * | 2018-12-18 | 2021-04-09 | 成都信息工程大学 | Spectral drift estimation method for satellite short-wave infrared hyperspectral observation data |
CN109871830A (en) * | 2019-03-15 | 2019-06-11 | 中国人民解放军国防科技大学 | Spatial-spectral fusion hyperspectral image classification method based on three-dimensional depth residual error network |
CN110135432A (en) * | 2019-05-24 | 2019-08-16 | 哈尔滨工程大学 | A kind of high-spectrum remote sensing dividing method based on K-means cluster |
CN110992245B (en) * | 2019-11-18 | 2023-07-25 | 沈阳航空航天大学 | Hyperspectral image dimension reduction method and device |
CN112199990B (en) * | 2020-08-27 | 2023-06-23 | 重庆工商大学 | Hyperspectral image segmentation system and method |
WO2023026742A1 (en) * | 2021-08-25 | 2023-03-02 | 浜松ホトニクス株式会社 | Dye image acquisition method, dye image acquisition device, and dye image acquisition program |
CN114119575B (en) * | 2021-11-30 | 2022-07-19 | 二十一世纪空间技术应用股份有限公司 | Spatial information change detection method and system |
CN116188423B (en) * | 2023-02-22 | 2023-08-08 | 哈尔滨工业大学 | Super-pixel sparse and unmixed detection method based on pathological section hyperspectral image |
CN116933187B (en) * | 2023-09-15 | 2023-12-19 | 北京中企导航科技有限公司 | Digital processing method and system for financial reimbursement bill |
Family Cites Families (44)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3217987B2 (en) | 1997-03-31 | 2001-10-15 | 松下電器産業株式会社 | Video signal decoding method and encoding method |
CA2219809A1 (en) * | 1997-10-31 | 1999-04-30 | Shen-En Qian | System for interactive visualization and analysis of imaging spectrometry datasets over a wide-area network |
US6208752B1 (en) * | 1998-03-12 | 2001-03-27 | The United States Of America As Represented By The Secretary Of The Navy | System for eliminating or reducing exemplar effects in multispectral or hyperspectral sensors |
US6075891A (en) | 1998-07-06 | 2000-06-13 | General Dynamics Government Systems Corporation | Non-literal pattern recognition method and system for hyperspectral imagery exploitation |
US6504943B1 (en) | 1998-07-20 | 2003-01-07 | Sandia Corporation | Information-efficient spectral imaging sensor |
US6952499B1 (en) | 1998-10-19 | 2005-10-04 | The United States Of America As Represented By The Secretary Of The Air Force | Aspire (autonomous spatial pattern identification and recognition algorithm) |
US6356646B1 (en) | 1999-02-19 | 2002-03-12 | Clyde H. Spencer | Method for creating thematic maps using segmentation of ternary diagrams |
US6282301B1 (en) | 1999-04-08 | 2001-08-28 | The United States Of America As Represented By The Secretary Of The Army | Ares method of sub-pixel target detection |
US7180588B2 (en) | 1999-04-09 | 2007-02-20 | Plain Sight Systems, Inc. | Devices and method for spectral measurements |
US6859275B2 (en) | 1999-04-09 | 2005-02-22 | Plain Sight Systems, Inc. | System and method for encoded spatio-spectral information processing |
US6665438B1 (en) * | 1999-05-05 | 2003-12-16 | American Gnc Corporation | Method for hyperspectral imagery exploitation and pixel spectral unmixing |
US6782138B1 (en) | 1999-07-14 | 2004-08-24 | Purdue Research Foundation | Method and system for processing multidimensional data |
US6750964B2 (en) | 1999-08-06 | 2004-06-15 | Cambridge Research And Instrumentation, Inc. | Spectral imaging methods and systems |
US6940999B2 (en) | 2000-06-13 | 2005-09-06 | American Gnc Corp. | Method for target detection and identification by using proximity pixel information |
US6763136B1 (en) | 2000-09-19 | 2004-07-13 | Bae Systems Mission Solutions, Inc. | Method and apparatus for determining spectral similarity |
US6804400B1 (en) | 2000-11-01 | 2004-10-12 | Bae Systems Mission Solutions Inc. | Adaptive hyperspectral data compression |
US6701021B1 (en) | 2000-11-22 | 2004-03-02 | Canadian Space Agency | System and method for encoding/decoding multidimensional data using successive approximation multi-stage vector quantization |
US6608931B2 (en) | 2001-07-11 | 2003-08-19 | Science Applications International Corporation | Method for selecting representative endmember components from spectral data |
US7035431B2 (en) | 2002-02-22 | 2006-04-25 | Microsoft Corporation | System and method for probabilistic exemplar-based pattern tracking |
US7200243B2 (en) | 2002-06-28 | 2007-04-03 | The United States Of America As Represented By The Secretary Of The Army | Spectral mixture process conditioned by spatially-smooth partitioning |
DE10354752B4 (en) | 2002-11-25 | 2006-10-26 | Deutsches Zentrum für Luft- und Raumfahrt e.V. | Method and device for the automatic equalization of single-channel or multi-channel images |
US20040218172A1 (en) | 2003-01-24 | 2004-11-04 | Deverse Richard A. | Application of spatial light modulators for new modalities in spectrometry and imaging |
US7551785B2 (en) | 2003-07-03 | 2009-06-23 | Canadian Space Agency | Method and system for compressing a continuous data flow in real-time using cluster successive approximation multi-stage vector quantization (SAMVQ) |
US7194111B1 (en) | 2003-07-10 | 2007-03-20 | The United States Of America As Represented By The Secretary Of The Navy | Hyperspectral remote sensing systems and methods using covariance equalization |
US7251376B2 (en) | 2003-08-29 | 2007-07-31 | Canadian Space Agency | Data compression engines and real-time wideband compressor for multi-dimensional data |
US7321791B2 (en) | 2003-09-23 | 2008-01-22 | Cambridge Research And Instrumentation, Inc. | Spectral imaging of deep tissue |
US7463778B2 (en) | 2004-01-30 | 2008-12-09 | Hewlett-Packard Development Company, L.P | Motion estimation for compressing multiple view images |
CA2631564A1 (en) | 2004-11-29 | 2006-06-01 | Hypermed, Inc. | Medical hyperspectral imaging for evaluation of tissue and tumor |
JP4755490B2 (en) | 2005-01-13 | 2011-08-24 | オリンパスイメージング株式会社 | Blur correction method and imaging apparatus |
US7680337B2 (en) | 2005-02-22 | 2010-03-16 | Spectral Sciences, Inc. | Process for finding endmembers in a data set |
JP4654773B2 (en) | 2005-05-31 | 2011-03-23 | 富士フイルム株式会社 | Information processing apparatus, moving image encoding apparatus, information processing method, and information processing program |
US7961957B2 (en) * | 2007-01-30 | 2011-06-14 | Alon Schclar | Diffusion bases methods for segmentation and clustering |
US8538195B2 (en) | 2007-09-17 | 2013-09-17 | Raytheon Company | Hyperspectral image dimension reduction system and method |
US8078009B2 (en) | 2008-07-08 | 2011-12-13 | Harris Corporation | Optical flow registration of panchromatic/multi-spectral image pairs |
US8150195B2 (en) | 2008-10-03 | 2012-04-03 | Sony Corporation | Adaptive prediction using a dimensionality reduction process |
CN101763445B (en) | 2008-12-23 | 2011-11-09 | 北京理工大学 | High-spectrum image dimensionality reduction chip |
US8203114B2 (en) * | 2009-05-14 | 2012-06-19 | Raytheon Company | Adaptive spatial-spectral processing (ASSP) |
US8315472B2 (en) | 2009-05-29 | 2012-11-20 | Raytheon Company | System and method for reducing dimensionality of hyperspectral images |
US8175393B2 (en) | 2010-07-26 | 2012-05-08 | Raytheon Company | Multi-phenomenology object detection |
US8675989B2 (en) * | 2011-04-13 | 2014-03-18 | Raytheon Company | Optimized orthonormal system and method for reducing dimensionality of hyperspectral images |
US8670620B2 (en) * | 2011-08-10 | 2014-03-11 | National Ict Australia Limited | Decomposing hyperspectral or multispectral image data |
US8842937B2 (en) * | 2011-11-22 | 2014-09-23 | Raytheon Company | Spectral image dimensionality reduction system and method |
US8515179B1 (en) * | 2012-02-10 | 2013-08-20 | Raytheon Company | System and method for hyperspectral image compression |
US8655091B2 (en) * | 2012-02-24 | 2014-02-18 | Raytheon Company | Basis vector spectral image compression |
-
2012
- 2012-08-03 US US13/566,914 patent/US8660360B1/en active Active
-
2013
- 2013-06-12 WO PCT/US2013/045422 patent/WO2014021996A1/en active Application Filing
- 2013-06-12 EP EP13732757.3A patent/EP2880596B1/en active Active
-
2015
- 2015-02-02 IL IL237045A patent/IL237045B/en active IP Right Grant
Non-Patent Citations (2)
Title |
---|
CHEIN-I CHANG ED - CHEIN-I CHANG: "AN OPTICAL REAL-TIME ADAPTIVE SPECTRAL IDENTIFICATION SYSTEM (ORASIS)", 6 April 2007, HYPERSPECTRAL DATA EXPLOITATION: THEORY AND APPLICATIONS, WILEY, HOBOKEN, NJ, PAGE(S) 77 - 106, ISBN: 978-0-471-74697-3, XP007922093 * |
DING C ET AL: "Adaptive dimension reduction for clustering high dimensional data", DATA MINING, 2002. PROCEEDINGS. 2002 IEEE INTERNATIONAL CONFERENCE ON MAEBASHI CITY, JAPAN 9-12 DEC. 2002, LOS ALAMITOS, CA, USA,IEEE COMPUT. SOC, US, 9 December 2002 (2002-12-09), pages 147 - 154, XP010805112, ISBN: 978-0-7695-1754-4, DOI: 10.1109/ICDM.2002.1183897 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110363712A (en) * | 2019-06-14 | 2019-10-22 | 江苏理工学院 | A kind of high spectrum image solution mixing method of sparse dual constraint |
CN110363712B (en) * | 2019-06-14 | 2023-06-23 | 江苏理工学院 | Sparse dual constraint hyperspectral image unmixing method |
CN115834895A (en) * | 2023-02-22 | 2023-03-21 | 创银科技(南通)有限公司 | Data efficient compression storage method for unmanned aerial vehicle |
CN115834895B (en) * | 2023-02-22 | 2023-12-15 | 上海每时每刻文化传播有限公司 | Efficient data compression and storage method for unmanned aerial vehicle |
Also Published As
Publication number | Publication date |
---|---|
IL237045B (en) | 2018-03-29 |
US8660360B1 (en) | 2014-02-25 |
EP2880596B1 (en) | 2018-08-15 |
EP2880596A1 (en) | 2015-06-10 |
US20140037209A1 (en) | 2014-02-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2880596B1 (en) | System and method for reduced incremental spectral clustering | |
US9147265B2 (en) | System and method for rapid cluster analysis of hyperspectral images | |
US8948540B2 (en) | Optimized orthonormal system and method for reducing dimensionality of hyperspectral images | |
Tang et al. | Multi-kernel correlation filter for visual tracking | |
JP6133030B2 (en) | Dimension reduction method for hyperspectral image data | |
WO2020160641A1 (en) | Shadow and cloud masking for remote sensing images in agriculture applications using multilayer perceptron | |
US9734424B2 (en) | Sensor data filtering | |
CN110751087B (en) | EOF-based unmanned aerial vehicle signal identification system and method | |
CN107341505B (en) | Scene classification method based on image significance and Object Bank | |
CN111680579B (en) | Remote sensing image classification method for self-adaptive weight multi-view measurement learning | |
CN114743009B (en) | Hyperspectral image band selection method and system and electronic equipment | |
CN115115855A (en) | Training method, device, equipment and medium for image encoder | |
US8918347B2 (en) | Methods and systems for computer-based selection of identifying input for class differentiation | |
Nyasaka et al. | Learning hyperspectral feature extraction and classification with resnext network | |
Weinmann et al. | Investigations on the potential of hyperspectral and Sentinel-2 data for land-cover/land-use classification | |
CN113343900A (en) | Combined nuclear remote sensing image target detection method based on combination of CNN and superpixel | |
Hamouda et al. | Framework for automatic selection of kernels based on convolutional neural networks and ckmeans clustering algorithm | |
CN111667495A (en) | Image scene analysis method and device | |
Islam et al. | Subgrouping-based nmf with imbalanced class handling for hyperspectral image classification | |
Shen et al. | A holistic image segmentation framework for cloud detection and extraction | |
Salloum et al. | Efficient image splicing localization via contrastive feature extraction | |
Mehta et al. | SPCA assisted correlation clustering of hyperspectral imagery | |
Chopra et al. | Summarization of Band Selection Methods For Hyperspectral Images | |
Amankwah | A Multivariate Gradient and Mutual Information Measure Method for Hyperspectral Image Visualization | |
Vogel | Transfer Learning for Hyperspectral Images Utilizing Channel Selection Techniques and Ensemble Methods |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13732757 Country of ref document: EP Kind code of ref document: A1 |
|
WWE | Wipo information: entry into national phase |
Ref document number: 237045 Country of ref document: IL |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2013732757 Country of ref document: EP |