CN113157661A - Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform - Google Patents

Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform Download PDF

Info

Publication number
CN113157661A
CN113157661A CN202110451095.9A CN202110451095A CN113157661A CN 113157661 A CN113157661 A CN 113157661A CN 202110451095 A CN202110451095 A CN 202110451095A CN 113157661 A CN113157661 A CN 113157661A
Authority
CN
China
Prior art keywords
matrix
data
hyperspectral
remote sensing
distributed parallel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110451095.9A
Other languages
Chinese (zh)
Inventor
孙晋
沈源
吴泽彬
徐洋
韦志辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN202110451095.9A priority Critical patent/CN113157661A/en
Publication of CN113157661A publication Critical patent/CN113157661A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24—Querying
    • G06F16/245—Query processing
    • G06F16/2453—Query optimisation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Image Processing (AREA)

Abstract

The invention discloses a cloud platform-based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data, which comprises the following steps of: uploading the hyperspectral original images to an HDFS (Hadoop distributed file system) in a blocking manner, and performing distributed parallel reading by using a Spark cloud computing framework; packaging the reading result into a matrix form, and executing a bilateral Jacobi method; and combining the calculation results of all the block data, and executing a bilateral Jacobi algorithm on the combined matrix. The method can be used for quickly and accurately performing singular value decomposition on the hyperspectral remote sensing image.

Description

Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform
Technical Field
The invention belongs to the technical field of distributed parallel processing of remote sensing images, and particularly relates to a distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on a cloud platform.
Background
The hyperspectral sensor obtains a near-continuous spectrum curve of a ground object by using dozens of even hundreds of narrow electromagnetic wave bands, and combines spectral information reflecting the reflection characteristic of the remote sensing object with images reflecting the spatial position relation of the remote sensing object. The hyperspectral image can provide rich information in three dimensions of space, spectrum and time, and since the 1980 s appeared, the hyperspectral remote sensing technology has been widely applied to various fields of mineral product measurement, precision agriculture, military and the like.
The hyperspectral remote sensing images are numerous in wave bands and contain abundant spectral information, but the correlation between adjacent spectra is high, so that the original hyperspectral images contain a large amount of redundant information, and therefore, the key information is extracted from massive hyperspectral remote sensing data. Scholars at home and abroad have made a great deal of research on this. Among them, Singular Value Decomposition (SVD) is widely used in hyperspectral remote sensing image processing. The singular value decomposition is a classical matrix decomposition method in linear algebra, the speed of the singular value reduction of the matrix is very fast under most conditions, the sum of the first 1-10% singular values is almost equal to the sum of all the singular values, and therefore the first k singular values and the corresponding singular matrix can be selected to carry out approximate expression on the original matrix. The singular value can reflect the inherent attribute of the hyperspectral image and plays an important role in the subsequent processing of the hyperspectral image.
The hyperspectral image singular value decomposition algorithm has the characteristics of high calculation complexity and large data volume, so that researchers utilize a high-performance parallel calculation technology to improve the calculation efficiency of the singular value decomposition algorithm, such as GPU (graphics processing unit), multi-core calculation and the like. However, these methods cannot solve the bottleneck problems of hard disk storage, single-machine memory and the like, the hyperspectral data volume increases in a geometric progression with the improvement of the hyperspectral image space and spectral resolution, the traditional single-machine processing mode is no longer suitable for the current massive data scale, and the data processing mode needs to be converted into a distributed parallel mode urgently. Spectral feature information is also the most direct information provided by hyperspectral images, which however have many bands and contain many redundant information. If the data is directly processed, the Hughes phenomenon is easy to occur, and the data processing is inconvenient. Therefore, a suitable method is needed to reduce the dimensionality of the hyperspectral data.
Disclosure of Invention
The invention aims to provide a high-speed and high-precision distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on a cloud platform.
The technical solution for realizing the purpose of the invention is as follows: a distributed parallel optimization method of singular value decomposition of hyperspectral remote sensing big data based on a cloud platform comprises the following steps:
uploading the hyperspectral original images to an HDFS (Hadoop distributed file system) in a blocking manner, and performing distributed parallel reading by using a Spark cloud computing framework;
packaging the read result into a matrix form, and executing a bilateral Jacobi algorithm;
and combining the calculation results of all the block data, and executing a bilateral Jacobi algorithm on the combined matrix.
As a preferred mode of the present invention, block uploading a hyperspectral original image to an HDFS distributed file system includes:
the data uploading process of the HDFS is that hyperspectral data is cut into a series of blocks according to set block size, then the blocks are stored in a distributed file system, and the block size needs to be set to be a proper value according to the size of a memory; for hyperspectral data, the data integrity of each block is ensured in the uploading process.
As a preferred mode of the present invention, blockSize is set to a suitable value according to the memory size, and the value is an integer multiple of Hadoop parameter checksum.
As a preferred mode of the present invention, the distributed parallel reading is performed by using a Spark cloud computing framework, including:
and calling a newAPIHadoopfile method by the Spark through a Spark context object to read the hyperspectral data stored on the HDFS to generate RDDs (RDDs), wherein each block corresponds to one partition in the RDDs, and the number of the fragments of the hyperspectral data is the parallelism of a program for processing the data by the Spark.
As a preferred mode of the invention, the RDD type is NewHadoop RDD, the RDD type is a key value pair RDD, key is the offset of block in the original hyperspectral data, value is Byte array, namely the original hyperspectral data, and the key is converted in a map operatorFor the offset of the first pixel in the block in the pixel vector of the original Data, value is converted into a two-dimensional array Data(rows×cols)×bandsWherein rows is the number of rows of the hyperspectral data, cols is the number of columns, bands is the number of bands, and rows × cols is the number of pixels.
As a preferred mode of the invention, the read result is encapsulated into a matrix form, and the bilateral Jacobi algorithm is executed, and the method comprises the following specific steps:
s201, for the hyperspectral data A epsilon Rm×nConstructing a symmetric matrix H ═ ATA, wherein m is the row number of the hyperspectral image, and n is the product of the column number of the hyperspectral image and the wave band number;
s202, initializing unit matrix V ═ In×n；
S203, finding an element H (i, j) with the maximum absolute value from the non-main diagonal elements of H;
s204, constructing a Jacobi rotation matrix
Figure BDA0003038699430000031
Wherein
Figure BDA0003038699430000032
The cosine value of the rotation angle of Jacobi,
Figure BDA0003038699430000033
sine value of Jacobi rotation angle, hiiDenotes the value of the element in the symmetric matrix H indexed as the ith row and ith column, HjjAnd hijThe same process is carried out;
s205, matrix H and matrix V are respectively corresponding to Jacobi rotation matrix UijMultiplying to obtain a result A1And updating H; obtaining a result V1And updating V;
s206, judging whether the maximum value of the non-main diagonal elements of the current matrix H is smaller than a threshold value e, if so, executing the next step, otherwise, returning to the step S203;
s207, multiplying the original matrix A by the calculation result V obtained in the step S206 to obtain a matrix U, and executing normalization operation, namely dividing each column vector of U by the two norms of the column vector;
and S208, performing appropriate column interception on the left singular matrix U, the singular value matrix sigma and the right singular matrix V of the calculation result obtained in the step S207 to obtain a final calculation result of the block data.
As a preferred mode of the present invention, the calculation results of all the block data are merged, and the bilateral Jacobi algorithm is executed on the merged matrix, which is specifically as follows:
s301, executing the calculation result collect of the bilateral Jacobi to a driver end on all the block data obtained in the last step, splicing the V-sigma matrix in the Svd calculation result of each block along the horizontal direction, splicing the U matrix along the diagonal direction, and obtaining a spliced matrix:
Figure BDA0003038699430000034
s302, transposing the splicing matrix M obtained in the step S301 to obtain a matrix MT，MTPerforming matrix multiplication operation with M to obtain matrix A ═ MTM, obtaining a characteristic vector matrix V' of the matrix A by a bilateral Jacob method;
s303, multiplying the splicing matrix M by the V', and updating the splicing matrix M by using the obtained result so that any column pair of M meets the orthogonality;
s304, multiplying the splicing matrix V by the splicing matrix V', and updating the splicing matrix V by using the obtained result;
s305, performing normalization operation on the matrix V, rearranging column vectors according to the size of singular values, and performing certain interception operation;
and S306, exchanging the positions of the left singular matrix M and the right singular matrix V.
The electronic equipment comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor executes the program to realize the cloud platform-based distributed parallel optimization method for the singular value decomposition of the hyperspectral remote sensing big data.
A computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the above-mentioned cloud platform-based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data.
Compared with the prior art, the invention has the remarkable advantages that: (1) and setting a reasonable block size according to a hyperspectral image storage format, cutting the original hyperspectral image into a series of blocks, uploading the blocks to a Hadoop Distributed File System (HDFS) for storage and management. According to the data organization modes corresponding to different hyperspectral image file formats, corresponding hyperspectral image reading classes are designed, the overhead brought by moving data is reduced, and the algorithm efficiency is improved; (2) aiming at a hyperspectral data organization mode, a bilateral Jacobi iteration Svd algorithm is adopted to improve the memory utilization rate of a computing node; (3) the small-sized matrix is transferred to each computing node, and a reasonable task decomposition and intermediate data storage structure is designed, so that the time consumption caused by the sum of data transmission and intermediate data shuffling is reduced.
The invention provides a cloud platform-based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data, which is described in detail below with reference to the accompanying drawings.
Drawings
FIG. 1 is a flow chart of a distributed parallel optimization method of singular value decomposition of hyperspectral remote sensing big data based on a cloud platform.
Fig. 2 shows the running time of the parallel algorithm under different parallelism.
Detailed Description
With reference to fig. 1, the invention discloses a cloud platform-based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data, which comprises the following specific processes:
step 1, uploading hyperspectral original images to an HDFS distributed file system in a blocking mode, and performing distributed parallel reading by using a Spark cloud computing framework, wherein the method specifically comprises the following steps:
because the amount of the hyperspectral image data is large, and in order to reduce the overhead caused by moving the data, the embodiment adopts an HDFS distributed file system to store the hyperspectral data. The data uploading process of the HDFS is that hyperspectral data is cut into a series of blocks according to set block size, then the blocks are stored in a distributed file system, the block size needs to be set to be a proper value according to the size of a memory (the block size needs to be an integral multiple of Hadoop parameter checksum, otherwise errors can be reported), meanwhile, in order to avoid data inclination, resources of each computing node are utilized in a balanced mode, and the data size stored by each block needs to be guaranteed to be close to each other as much as possible. For hyperspectral data, the data integrity of each block is ensured in the uploading process. Taking the hyperspectral data in the BIP format as an example, because the hyperspectral data are stored in a pixel sequence, the blockSize is set to be an integral multiple of the size occupied by all the waveband data of one pixel.
And calling a newAPIHadoopfile method by the Spark through a Spark context object to read the hyperspectral data stored on the HDFS to generate RDDs (RDDs), wherein each block corresponds to one partition in the RDDs, and the number of the fragments of the hyperspectral data is the parallelism of a program for processing the data by the Spark. The RDD type is NewHadoop RDD, the RDD type is a key value pair RDD, key is the offset of the block in the original hyperspectral Data, value is a Byte array, namely the original hyperspectral Data, the key is converted into the offset of the first pixel in the block in the pixel vector of the original Data in a map operator, and the value is converted into a two-dimensional array Data(rows×cols)×bandsWherein rows is the number of rows of the hyperspectral data, cols is the number of columns, bands is the number of bands, and rows × cols is the number of pixels.
Step 2, encapsulating the reading result of the step 1 into a matrix form, and executing a bilateral Jacobi method, which comprises the following specific steps:
(2.1) for the hyperspectral data A ∈ Rm×nConstructing a symmetric matrix H ═ ATA；
(2.2) initializing the identity matrix V ═ In×n；
(2.3) finding an element H (i, j) with the maximum absolute value from the non-main diagonal elements of H;
(2.4) constructing Jacobi rotation matrix
Figure BDA0003038699430000051
Wherein
Figure BDA0003038699430000052
The cosine value of the rotation angle of Jacobi,
Figure BDA0003038699430000053
sine value of Jacobi rotation angle, hiiDenotes the value of the element in the symmetric matrix H indexed as the ith row and ith column, HjjAnd hijThe same process is carried out;
(2.5) matrix H and matrix V are respectively associated with Jacobi rotation matrix UijMultiplying to obtain a result H1And updating H; obtaining a result V1And updating V;
(2.6) judging whether the maximum value of the non-main diagonal elements of the current matrix H is smaller than a threshold value e, if so, executing the next step, otherwise, returning to the step (2.3);
(2.7) multiplying the original matrix A by the calculation result V obtained in the step (2.6) to obtain a matrix U, and performing normalization operation, namely dividing each column vector of U by the two norms of the column vector;
and (2.8) carrying out appropriate column interception on the left singular matrix U, the singular value matrix sigma and the right singular matrix V of the calculation result obtained in the step (2.7) to obtain a final calculation result of the block data.
And 3, combining the calculation results of all the block data, and executing a bilateral Jacobi algorithm on the combined matrix, wherein the method specifically comprises the following steps:
(3.1) executing the calculation result of bilateral Jacobi to a driver end on all the block data obtained in the step 2, splicing the V-sigma matrix in the Svd calculation result of each block along the horizontal direction, splicing the U matrix along the diagonal direction, and obtaining a splicing matrix:
Figure BDA0003038699430000061
(3.2) transposing the splicing matrix M obtained in the step (3.1) to obtain a matrix MT，MTAnd M execution momentThe array multiplication operation results in the matrix A being MTM, obtaining a characteristic vector matrix V' of the matrix A by a bilateral Jacob method;
(3.3) multiplying the splicing matrix M by V', and updating the splicing matrix M by using the obtained result so that any column pair of M meets the orthogonality property;
(3.4) multiplying the splicing matrix V by V', and updating the splicing matrix V by using the obtained result;
(3.5) carrying out normalization operation on the matrix V, rearranging column vectors according to the size of singular values, and carrying out certain interception operation;
(3.6) exchanging the positions of the left singular matrix M and the right singular matrix V;
in conclusion, the method is different from the traditional singular value decomposition method of the single-machine remote sensing image, and distributed parallel optimization based on Spark is performed on the bilateral JacobiSVD algorithm. In order to reduce the overhead brought by moving data and improve the data reading speed, the algorithm reasonably designs the storage of the hyperspectral image data, designs a distributed reading strategy of the hyperspectral image, divides the logic of the algorithm, executes the operation with larger calculation amount in a distributed parallel mode, optimizes the calculation logic of the bilateral Jacobi method and reduces a large amount of unnecessary calculation.
TABLE 1 comparison of singular values found by the Serial Algorithm and Spark parallel Algorithm
Figure BDA0003038699430000071
The experimental results shown in table 1 and fig. 2 can be used, and the speed of the optimized algorithm is obviously improved on the basis of ensuring the accuracy of the singular value decomposition result.

Claims (9)

1. A distributed parallel optimization method of hyperspectral remote sensing big data singular value decomposition based on a cloud platform is characterized by comprising the following steps:
uploading the hyperspectral original images to an HDFS (Hadoop distributed file system) in a blocking manner, and performing distributed parallel reading by using a Spark cloud computing framework;
packaging the read result into a matrix form, and executing a bilateral Jacobi algorithm;
and combining the calculation results of all the block data, and executing a bilateral Jacobi algorithm on the combined matrix.
2. The cloud platform based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data according to claim 1, wherein block uploading of hyperspectral original images to an HDFS distributed file system comprises:
the data uploading process of the HDFS is that hyperspectral data is cut into a series of blocks according to set block size, then the blocks are stored in a distributed file system, and the block size needs to be set to be a proper value according to the size of a memory; for hyperspectral data, the data integrity of each block is ensured in the uploading process.
3. The cloud platform based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data according to claim 2, wherein blockSize is set to a suitable value according to the size of a memory, and the value is an integral multiple of Hadoop parameters checksum.
4. The cloud platform based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data according to claim 2, which is characterized in that a Spark cloud computing framework is used for distributed parallel reading, and comprises the following steps:
and calling a newAPIHadoopfile method by the Spark through a Spark context object to read the hyperspectral data stored on the HDFS to generate RDDs (RDDs), wherein each block corresponds to one partition in the RDDs, and the number of the fragments of the hyperspectral data is the parallelism of a program for processing the data by the Spark.
5. The cloud platform based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data, according to claim 4, characterized in that the RDD type is NewHadoop RDD, which is a key value pair RDD, and key is block at original heightThe offset in the spectral Data, value is a Byte array, namely original hyperspectral Data, key is converted into the offset of the first pixel in the block in the pixel vector of the original Data in a map operator, and value is converted into a two-dimensional array Data(rows×cola)×bandsWherein rows is the number of rows of the hyperspectral data, cols is the number of columns, bands is the number of bands, and rows × cols is the number of pixels.
6. The cloud platform based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data according to claim 1 is characterized in that read results are packaged into a matrix form, and a bilateral Jacobi algorithm is executed, and the method comprises the following specific steps:
s201, for the hyperspectral data A epsilon Rm×nConstructing a symmetric matrix H ═ ATA, wherein m is the row number of the hyperspectral image, and n is the product of the column number of the hyperspectral image and the wave band number;
s202, initializing unit matrix V ═ In×n；
S203, finding an element H (i, j) with the maximum absolute value from the non-main diagonal elements of H;
s204, constructing a Jacobi rotation matrix
Figure FDA0003038699420000021
Wherein
Figure FDA0003038699420000022
Figure FDA0003038699420000023
The cosine value of the rotation angle of Jacobi,
Figure FDA0003038699420000024
sine value of Jacobi rotation angle, hiiDenotes the value of the element in the symmetric matrix H indexed as the ith row and ith column, HjjAnd hijThe same process is carried out;
s205, matrix H and matrix V are respectively corresponding to Jacobi rotation matrix UijMultiplying to obtain a result H1And updating H; obtaining a result V1And updating V;
s206, judging whether the maximum value of the non-main diagonal elements of the current matrix H is smaller than a threshold value e, if so, executing the next step, otherwise, returning to the step S203;
s207, multiplying the original matrix A by the calculation result V obtained in the step S206 to obtain a matrix U, and executing normalization operation, namely dividing each column vector of U by the two norms of the column vector;
and S208, performing appropriate column interception on the left singular matrix U, the singular value matrix sigma and the right singular matrix V of the calculation result obtained in the step S207 to obtain a final calculation result of the block data.
7. The cloud platform based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data according to claim 1 is characterized in that the calculation results of all block data are merged, and a bilateral Jacobi algorithm is executed on the merged matrix, specifically as follows:
s301, executing the calculation result collect of the bilateral Jacobi to a driver end on all the block data obtained in the last step, splicing the V-sigma matrix in the Svd calculation result of each block along the horizontal direction, splicing the U matrix along the diagonal direction, and obtaining a spliced matrix:
Figure FDA0003038699420000031
s302, transposing the splicing matrix M obtained in the step S301 to obtain a matrix MT，MTPerforming matrix multiplication operation with M to obtain matrix A ═ MTM, obtaining a characteristic vector matrix V' of the matrix A by a bilateral Jacob method;
s303, multiplying the splicing matrix M by the V', and updating the splicing matrix M by using the obtained result so that any column pair of M meets the orthogonality;
s304, multiplying the splicing matrix V by the splicing matrix V', and updating the splicing matrix V by using the obtained result;
s305, performing normalization operation on the matrix V, rearranging column vectors according to the size of singular values, and performing certain interception operation;
and S306, exchanging the positions of the left singular matrix M and the right singular matrix V.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the distributed parallel optimization method for cloud platform-based singular value decomposition of hyperspectral remote sensing big data when executing the program according to any one of claims 1 to 7.
9. A computer-readable storage medium, on which a computer program is stored, wherein the program, when executed by a processor, implements the cloud platform-based distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data.
CN202110451095.9A 2021-04-26 2021-04-26 Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform Pending CN113157661A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110451095.9A CN113157661A (en) 2021-04-26 2021-04-26 Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110451095.9A CN113157661A (en) 2021-04-26 2021-04-26 Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform

Publications (1)

Publication Number Publication Date
CN113157661A true CN113157661A (en) 2021-07-23

Family

ID=76870679

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110451095.9A Pending CN113157661A (en) 2021-04-26 2021-04-26 Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform

Country Status (1)

Country Link
CN (1) CN113157661A (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103532890A (en) * 2013-10-29 2014-01-22 东南大学 SVD method for complex channel matrix

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103532890A (en) * 2013-10-29 2014-01-22 东南大学 SVD method for complex channel matrix

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
呼思乐: "基于MapReduce的奇异值分解及PCA算法应用研究", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 *
陈星宇: "分布式高阶奇异值分解算法的设计及实现", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 *

Similar Documents

Publication Publication Date Title
Tang et al. Torchsparse: Efficient point cloud inference engine
US9478002B2 (en) Vertex parameter data compression
CN108805266A (en) A kind of restructural CNN high concurrents convolution accelerator
US11989638B2 (en) Convolutional neural network accelerating device and method with input data conversion
Lee et al. Efficient local statistical analysis via integral histograms with discrete wavelet transform
WO2020047823A1 (en) Convolution over sparse and quantization neural networks
CN108170639B (en) Tensor CP decomposition implementation method based on distributed environment
WO2022037257A1 (en) Convolution calculation engine, artificial intelligence chip, and data processing method
CN110796235B (en) Vectorization implementation method for Valid convolution of convolutional neural network
KR20190062303A (en) Method and apparatus for performing convolution operation on folded feature date
US20180373677A1 (en) Apparatus and Methods of Providing Efficient Data Parallelization for Multi-Dimensional FFTs
Li et al. FPGA implementation of a maximum simplex volume algorithm for endmember extraction from remotely sensed hyperspectral images
US11481994B2 (en) Method and apparatus for extracting image data in parallel from multiple convolution windows, device, and computer-readable storage medium
Netzel et al. Comparison of different implementations of a raster map calculator
CN114003201A (en) Matrix transformation method and device and convolutional neural network accelerator
Yzelman Generalised vectorisation for sparse matrix: vector multiplication
CN113157661A (en) Distributed parallel optimization method for singular value decomposition of hyperspectral remote sensing big data based on cloud platform
US20170337663A1 (en) Method and system for interpolating data
Gündoğar et al. Block tridiagonal matrix enhanced multivariance products representation (BTMEMPR)
CN115170381A (en) Visual SLAM acceleration system and method based on deep learning
Reinbold et al. Parameterized splitting of summed volume tables
WO2023046001A1 (en) Method and apparatus for matrix computation acceleration
Singla Raptor: large scale processing of big raster+ vector data
US20240086719A1 (en) Sparse encoding and decoding at mixture-of-experts layer
Noor et al. Parallel implementation of lossless clustered integer KLT using OpenMP

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination