CN111010191A - Data acquisition method, system, device and storage medium - Google Patents
Data acquisition method, system, device and storage medium Download PDFInfo
- Publication number
- CN111010191A CN111010191A CN201911271726.8A CN201911271726A CN111010191A CN 111010191 A CN111010191 A CN 111010191A CN 201911271726 A CN201911271726 A CN 201911271726A CN 111010191 A CN111010191 A CN 111010191A
- Authority
- CN
- China
- Prior art keywords
- data
- matrix
- acquisition
- acquired
- dimensional
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 238000005070 sampling Methods 0.000 claims abstract description 16
- 238000007906 compression Methods 0.000 claims abstract description 12
- 230000006835 compression Effects 0.000 claims abstract description 12
- 230000009467 reduction Effects 0.000 claims abstract description 10
- 239000011159 matrix material Substances 0.000 claims description 95
- 238000005259 measurement Methods 0.000 claims description 33
- 230000009466 transformation Effects 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 16
- 238000000354 decomposition reaction Methods 0.000 claims description 10
- 238000005457 optimization Methods 0.000 claims description 8
- 238000000844 transformation Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 230000001131 transforming effect Effects 0.000 claims description 4
- 239000013598 vector Substances 0.000 claims description 4
- 238000005516 engineering process Methods 0.000 abstract description 11
- 230000005540 biological transmission Effects 0.000 description 5
- 238000004891 communication Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000013480 data collection Methods 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009472 formulation Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000036632 reaction speed Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
- H03M7/00—Conversion of a code where information is represented by a given sequence or number of digits to a code where the same, similar or subset of information is represented by a different sequence or number of digits
- H03M7/30—Compression; Expansion; Suppression of unnecessary data, e.g. redundancy reduction
- H03M7/3059—Digital compression and data reduction techniques where the original information is represented by a subset or similar information, e.g. lossy compression
Abstract
The embodiment of the invention provides a data acquisition method, a system, equipment and a storage medium. The method comprises the following steps: acquiring data to be acquired; sampling the data to be acquired, and determining acquisition algorithm parameters; based on the acquisition algorithm parameters, carrying out compression reduction on the low-dimensional signals of the data to be acquired to obtain first original data; compressing and restoring the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data; and comparing the acquisition algorithm parameters with the second original data, and outputting to obtain compressed acquisition data. According to the embodiment of the invention, by adopting a compressed sensing technology and utilizing the space-time correlation and sparsity among data, the acquisition amount of sensor data can be greatly reduced, the burden of an energy internet for acquiring and transmitting massive underlying new energy data is reduced, and the overall performance of the system is improved.
Description
Technical Field
The present invention relates to the field of data acquisition technologies, and in particular, to a data acquisition method, system, device, and storage medium.
Background
The energy internet is a novel energy system, in order to fully and effectively access a large amount of distributed renewable energy and ensure the stability and robust operation of the system, the deep fusion of an information technology and a physical technology needs to be realized, the control performance and the reaction speed of the system are improved through fusion, the accurate perception of the network operation state is realized, and the robustness, timeliness and accuracy of system strategy formulation are ensured.
Information and physics in the energy internet cannot be fused with each other to acquire bottom-layer data, and in order to realize comprehensive and accurate sensing of the system state and efficient utilization of new energy, a large number of sensors are generally required to be deployed in the system range aiming at the same or similar new energy facilities so as to acquire sample data through the sensors.
In the prior art, a large number of sample points need to be collected in the data collection process, so that the collection, storage and transmission pressure is overlarge.
Disclosure of Invention
The embodiment of the invention provides a data acquisition method, a system, equipment and a storage medium, which are used for solving the defect that in the prior art, a large number of sample points are required to be acquired in the data acquisition process, so that the acquisition, storage and transmission pressure is overlarge.
In a first aspect, an embodiment of the present invention provides a data acquisition method, including:
acquiring data to be acquired;
sampling the data to be acquired, and determining acquisition algorithm parameters;
based on the acquisition algorithm parameters, carrying out compression reduction on the low-dimensional signals of the data to be acquired to obtain first original data;
compressing and restoring the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data;
and comparing the acquisition algorithm parameters with the second original data, and outputting to obtain compressed acquisition data.
The sampling of the data to be acquired and the determination of the acquisition algorithm parameters specifically include:
and determining that the number of distributed nodes of the data to be acquired is M, the dimensionality of the low-dimensional data is N, and the dimensionality of the high-dimensional data is N.
The method is characterized in that based on the acquisition algorithm parameters, the low-dimensional signal of the data to be acquired is compressed and restored to obtain first original data, and the method specifically comprises the following steps:
obtaining a first measurement matrix as { Ψ }m1,nRespectively collecting m1 pieces of low-dimensional data from each node in the distributed nodes according to the first measurement matrix to form a first data matrix
Obtaining sparse transforms
Combining the first data matrix and the first measurement matrix to obtain a relational expression:
wherein s is a first sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining the first original data with n dimensions
Based on the first original data, compressing and restoring the high-dimensional signal of the data to be acquired to obtain second original data, which specifically comprises:
solving a covariance matrix C for the first raw data x, and solving the following eigenvalue decomposition problem to obtain a new set of eigen transformations
Where lambda denotes a diagonal matrix with eigenvalues as diagonals,
an n-dimensional feature transformation matrix composed of feature vectors obtained by feature decomposition;
transforming the n-dimensional feature into a matrix
Interpolating to high-dimensional space to obtain N-dimensional feature transformation matrix
Obtaining a second measurement matrix as { Ψ }m2,NRespectively collecting m2 pieces of high-dimensional data from each node in the distributed nodes according to the second measurement matrix to form a second data matrix
And combining the second data matrix, the second measurement matrix and the N-dimensional feature transformation matrix to obtain a relational expression based on the second data matrix:
wherein S is a second sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining second original data of N dimensions
Comparing the acquisition algorithm parameters with the second original data, and outputting to obtain compressed acquisition data, wherein the method specifically comprises the following steps:
comparing the distributed node number M in the acquisition algorithm with the second original data
Reducing to obtain M compressed collected data
Wherein the first measurement matrix and the second measurement matrix comprise the following matrices: gaussian random matrix, bernoulli random matrix and partially orthogonal matrix.
Wherein the sparse transform comprises: discrete cosine transform, fourier transform, and discrete wavelet transform.
In a second aspect, an embodiment of the present invention provides a data acquisition system based on multiple distributed compressed sensing, including:
the acquisition module is used for acquiring data to be acquired;
the sampling module is used for sampling the data to be acquired and determining acquisition algorithm parameters;
the first processing module is used for compressing and restoring the low-dimensional signal of the data to be acquired based on the acquisition algorithm parameters to obtain first original data;
the second processing module is used for compressing and restoring the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data;
and the output module is used for comparing the acquisition algorithm parameters with the second original data and outputting to obtain compressed acquisition data.
In a third aspect, an embodiment of the present invention provides an electronic device, including:
the data acquisition method based on the multiple distributed compressed sensing comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the steps of any one of the multiple distributed compressed sensing-based data acquisition methods.
In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements any one of the steps of the multiple distributed compressed sensing-based data acquisition method.
According to the technical scheme provided by the embodiment of the invention, by means of a compressed sensing technology and by means of the space-time correlation and sparsity among data, the acquisition amount of sensor data can be greatly reduced, the burden of an energy internet for acquiring and transmitting massive underlying new energy data is reduced, and the overall performance of the system is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 is a flowchart of a data acquisition method based on multiple distributed compressed sensing according to an embodiment of the present invention;
fig. 2 is a structural diagram of a data acquisition system based on multiple distributed compressed sensing according to an embodiment of the present invention;
fig. 3 is a block diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The compressed sensing technology is a typical technology in the information field, can realize accurate and effective recovery of system state data by utilizing data sparsity, greatly reduces the amount of sampled data, and saves the system cost. In the energy internet, if the acquired massive new energy data is not effectively compressed, huge data acquisition, storage and processing overhead is brought, service realization time delay is influenced, and great influence is brought to real-time control and immediate operation of the system. How to adopt an efficient compression algorithm is a challenging problem, and the acquisition amount of new energy data is reduced while the system state is accurately sensed. Aiming at the problem, the traditional compressed sensing algorithm is further improved, the sparsity expression of data is improved based on the combination of high-low dimension compressed sensing, and the data collection amount is effectively reduced.
The basic idea of the distributed compressed sensing method is that the space-time correlation of data in distributed nodes of the Internet of things is utilized, sample information is collected at each node respectively through the compressed sensing method and transmitted to a sink node for unified reduction processing, and therefore the node sample collection pressure can be reduced. The multi-distributed compressed sensing method provided by the embodiment of the invention comprises the steps of firstly collecting low-dimensional node information, reducing low-dimensional signals by using a traditional compression reduction algorithm, then solving a covariance matrix of each node signal, obtaining a group of new characteristic transformations by characteristic decomposition and transformation matrix interpolation of covariance, and then reducing high-dimensional signals by using distributed compressed sensing under the group of new characteristic transformations. Because the sparsity of the signals is further improved under the expression of the new characteristic basis functions, the number of sample points required to be collected for restoration can be further reduced, and the pressure of data on collection, storage and transportation is further reduced.
Therefore, the embodiment of the invention provides a data acquisition method based on multiple distributed compressed sensing.
Fig. 1 is a flowchart of a data acquisition method based on multiple distributed compressed sensing according to an embodiment of the present invention, as shown in fig. 1, including:
s1, acquiring data to be acquired;
s2, sampling the data to be collected, and determining collection algorithm parameters;
s3, based on the acquisition algorithm parameters, carrying out compression reduction on the low-dimensional signals of the data to be acquired to obtain first original data;
s4, based on the first original data, compressing and restoring the high-dimensional signal of the data to be acquired to obtain second original data;
and S5, comparing the acquisition algorithm parameters with the second original data, and outputting to obtain compressed acquisition data.
Specifically, data to be acquired and processed are acquired, the data are sampled, relevant parameters of an adopted acquisition algorithm, such as the number of sampling points, the used data dimension and other parameter information, are determined, after the relevant parameters of the acquisition algorithm are determined, a low-dimensional signal of the data to be acquired is compressed and restored to obtain first original data, further conversion processing is performed on the basis of the obtained first original data to obtain input parameters of a high-dimensional signal, the high-dimensional signal of the acquired data is further compressed and restored to obtain second original data, and finally, the final compressed acquired data are obtained by referring to the relevant parameters which are determined and the obtained second original data.
According to the embodiment of the invention, by adopting a compressed sensing technology and utilizing the space-time correlation and sparsity among data, the acquisition amount of sensor data can be greatly reduced, the burden of an energy internet for acquiring and transmitting massive underlying new energy data is reduced, and the overall performance of the system is improved.
On the basis of the foregoing embodiment, step S2 in the method specifically includes:
and determining that the number of distributed nodes of the data to be acquired is M, the dimensionality of the low-dimensional data is N, and the dimensionality of the high-dimensional data is N.
On the basis of the foregoing embodiment, step S3 in the method specifically includes:
obtaining a first measurement matrix as { Ψ }m1,nRespectively collecting m1 pieces of low-dimensional data from each node in the distributed nodes according to the first measurement matrix to form a first data matrix
Obtaining sparse transforms
Combining the first data matrix and the first measurement matrix to obtain a relational expression:
wherein s is a first sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining the first original data with n dimensions
The algorithm here is a low-dimensional signal compression reduction algorithm.
On the basis of the foregoing embodiment, step S4 in the method specifically includes:
solving a covariance matrix C for the first raw data x, and solving the following eigenvalue decomposition problem to obtain a new set of eigen transformations
Where lambda denotes a diagonal matrix with eigenvalues as diagonals,
an n-dimensional feature transformation matrix composed of feature vectors obtained by feature decomposition;
transforming the n-dimensional feature into a matrix
Interpolating to high-dimensional space to obtain N-dimensional feature transformation matrix
Obtaining a second measurement matrix as { Ψ }m2,NRespectively collecting m2 pieces of high-dimensional data from each node in the distributed nodes according to the second measurement matrix to form a second data matrix
And combining the second data matrix, the second measurement matrix and the N-dimensional feature transformation matrix to obtain a relational expression based on the second data matrix:
wherein S is a second sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining second original data of N dimensions
Specifically, the algorithm is a high-dimensional signal compression reduction algorithm, and before reducing high-dimensional data, the algorithm needs to obtain a group of problem-dependent feature transformations by using feature decomposition of covariance of low-dimensional data, and then obtains the high-dimensional feature transformations by interpolation.
According to the embodiment of the invention, the high-low-dimensional compressed sensing problem is realized in a serial manner, the effective compression of the energy internet space-time related new energy collection data is realized, and the data transmission and storage expenses are reduced.
On the basis of the foregoing embodiment, step S5 specifically includes:
comparing the distributed node number M in the acquisition algorithm with the second original data
Reducing to obtain M compressed collected data
On the basis of the above embodiment, the first measurement matrix and the second measurement matrix include the following matrices: gaussian random matrix, bernoulli random matrix and partially orthogonal matrix.
Specifically, the measurement matrix can be selected in various ways, and a gaussian random matrix, a bernoulli random matrix, a partially orthogonal matrix, or the like can be used.
On the basis of the above embodiment, the sparse transform includes: discrete cosine transform, fourier transform, and discrete wavelet transform.
Specifically, the sparse transform matrix can have different selection forms according to different types of collected data, and the classical sparse transform includes Discrete Cosine Transform (DCT), fourier transform (FFT), Discrete Wavelet Transform (DWT), and the like.
According to the embodiment of the invention, by adopting a compressed sensing technology and utilizing the space-time correlation and sparsity among data, the acquisition amount of sensor data can be greatly reduced, the burden of an energy internet for acquiring and transmitting massive underlying new energy data is reduced, and the overall performance of the system is improved.
Fig. 2 is a structural diagram of a data acquisition system based on multiple distributed compressed sensing according to an embodiment of the present invention, as shown in fig. 2, including: the device comprises an acquisition module 21, a sampling module 22, a first processing module 23, a second processing module 24 and an output module 25; wherein:
the acquisition module 21 is used for acquiring data to be acquired; the sampling module 22 is configured to sample the data to be acquired and determine an acquisition algorithm parameter; the first processing module 23 is configured to compress and restore the low-dimensional signal of the data to be acquired based on the acquisition algorithm parameter to obtain first original data; the second processing module 24 is configured to compress and restore the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data; the output module 25 is configured to compare the acquisition algorithm parameter with the second raw data, and output the comparison result to obtain compressed acquisition data.
The system provided by the embodiment of the present invention is used for executing the corresponding method, the specific implementation manner of the system is consistent with the implementation manner of the method, and the related algorithm flow is the same as the algorithm flow of the corresponding method, which is not described herein again.
According to the embodiment of the invention, by adopting a compressed sensing technology and utilizing the space-time correlation and sparsity among data, the acquisition amount of sensor data can be greatly reduced, the burden of an energy internet for acquiring and transmitting massive underlying new energy data is reduced, and the overall performance of the system is improved.
On the basis of the foregoing embodiment, the sampling module 22 is specifically configured to:
and determining that the number of distributed nodes of the data to be acquired is M, the dimensionality of the low-dimensional data is N, and the dimensionality of the high-dimensional data is N.
On the basis of the foregoing embodiment, the first processing module 23 is specifically configured to:
obtaining a first measurement matrix as { Ψ }m1,nRespectively collecting m1 pieces of low-dimensional data from each node in the distributed nodes according to the first measurement matrix to form a first data matrix
Obtaining sparse transforms
Combining the first data matrix and the first measurement matrix to obtain a relational expression:
wherein s is a first sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining the first original data with n dimensions
On the basis of the foregoing embodiment, the second processing module 24 is specifically configured to:
solving a covariance matrix C for the first raw data x, and solving the following eigenvalue decomposition problem to obtain a new set of eigen transformations
Where lambda denotes a diagonal matrix with eigenvalues as diagonals,
an n-dimensional feature transformation matrix composed of feature vectors obtained by feature decomposition;
transforming the n-dimensional feature into a matrix
Interpolating to high-dimensional space to obtain N-dimensional feature transformation matrix
Obtaining a second measurement matrix as { Ψ }m2,NRespectively collecting m2 pieces of high-dimensional data from each node in the distributed nodes according to the second measurement matrix to form a second data matrix
And combining the second data matrix, the second measurement matrix and the N-dimensional feature transformation matrix to obtain a relational expression based on the second data matrix:
wherein S is a second sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining second original data of N dimensions
According to the embodiment of the invention, the high-low-dimensional compressed sensing problem is realized in a serial manner, the effective compression of the energy internet space-time related new energy collection data is realized, and the data transmission and storage expenses are reduced.
On the basis of the foregoing embodiment, the output module 25 is specifically configured to:
comparing the distributed node number M in the acquisition algorithm with the second original data
Reducing to obtain M compressed collected data
On the basis of the above embodiment, the first measurement matrix in the first processing module 23 and the second measurement matrix in the second processing module 24 include the following matrices: gaussian random matrix, bernoulli random matrix and partially orthogonal matrix.
On the basis of the above embodiment, the sparse transform in the first processing module 23 includes: discrete cosine transform, fourier transform, and discrete wavelet transform.
According to the embodiment of the invention, by adopting a compressed sensing technology and utilizing the space-time correlation and sparsity among data, the acquisition amount of sensor data can be greatly reduced, the burden of an energy internet for acquiring and transmitting massive underlying new energy data is reduced, and the overall performance of the system is improved.
Fig. 3 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 3: a processor (processor)310, a communication Interface (communication Interface)320, a memory (memory)330 and a communication bus 340, wherein the processor 310, the communication Interface 320 and the memory 330 communicate with each other via the communication bus 340. The processor 310 may call logic instructions in the memory 330 to perform the following method: acquiring data to be acquired; sampling the data to be acquired, and determining acquisition algorithm parameters; based on the acquisition algorithm parameters, carrying out compression reduction on the low-dimensional signals of the data to be acquired to obtain first original data; compressing and restoring the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data; and comparing the acquisition algorithm parameters with the second original data, and outputting to obtain compressed acquisition data.
In addition, the logic instructions in the memory 330 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, an embodiment of the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program is implemented to perform the transmission method provided in the foregoing embodiments when executed by a processor, and for example, the method includes: acquiring data to be acquired; sampling the data to be acquired, and determining acquisition algorithm parameters; based on the acquisition algorithm parameters, carrying out compression reduction on the low-dimensional signals of the data to be acquired to obtain first original data; compressing and restoring the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data; and comparing the acquisition algorithm parameters with the second original data, and outputting to obtain compressed acquisition data.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method of data acquisition, comprising:
acquiring data to be acquired;
sampling the data to be acquired, and determining acquisition algorithm parameters;
based on the acquisition algorithm parameters, carrying out compression reduction on the low-dimensional signals of the data to be acquired to obtain first original data;
compressing and restoring the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data;
and comparing the acquisition algorithm parameters with the second original data, and outputting to obtain compressed acquisition data.
2. The data acquisition method according to claim 1, wherein the sampling the data to be acquired and determining acquisition algorithm parameters specifically comprises:
and determining that the number of distributed nodes of the data to be acquired is M, the dimensionality of the low-dimensional data is N, and the dimensionality of the high-dimensional data is N.
3. The data acquisition method according to claim 2, wherein the compressing and restoring the low-dimensional signal of the data to be acquired based on the acquisition algorithm parameter to obtain first original data specifically includes:
obtaining a first measurement matrix as { Ψ }m1,nRespectively collecting m1 pieces of low-dimensional data from each node in the distributed nodes according to the first measurement matrix to form a first data matrix
Obtaining sparse transforms
Combining the first data matrix and the first measurement matrix to obtain a relational expression:
wherein s is a first sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining the first original data with n dimensions
4. The data acquisition method according to claim 3, wherein compressing and restoring the high-dimensional signal of the data to be acquired based on the first raw data to obtain second raw data specifically includes:
solving a covariance matrix C for the first raw data x, and solving the following eigenvalue decomposition problem to obtain a new set of eigen transformations
Where lambda denotes a diagonal matrix with eigenvalues as diagonals,
an n-dimensional feature transformation matrix composed of feature vectors obtained by feature decomposition;
transforming the n-dimensional feature into a matrix
Interpolating to high-dimensional space to obtain N-dimensional feature transformation matrix
Obtaining a second measurement matrix as { Ψ }m2,NFromRespectively collecting m2 pieces of high-dimensional data in each node of the distributed nodes according to the second measurement matrix to form a second data matrix
And combining the second data matrix, the second measurement matrix and the N-dimensional feature transformation matrix to obtain a relational expression based on the second data matrix:
wherein S is a second sparse representation coefficient;
obtaining s by solving the following optimization problem, and further obtaining second original data of N dimensions
5. The data acquisition method according to claim 4, wherein the outputting of the compressed acquisition data by comparing the acquisition algorithm parameters with the second raw data specifically comprises:
comparing the distributed node number M in the acquisition algorithm with the second original data
Reducing to obtain M compressed collected data
6. The data acquisition method of claim 4, wherein the first measurement matrix and the second measurement matrix comprise the following matrices: gaussian random matrix, bernoulli random matrix and partially orthogonal matrix.
7. The data acquisition method of claim 4, wherein the sparse transform comprises: discrete cosine transform, fourier transform, and discrete wavelet transform.
8. A data acquisition system, comprising:
the acquisition module is used for acquiring data to be acquired;
the sampling module is used for sampling the data to be acquired and determining acquisition algorithm parameters;
the first processing module is used for compressing and restoring the low-dimensional signal of the data to be acquired based on the acquisition algorithm parameters to obtain first original data;
the second processing module is used for compressing and restoring the high-dimensional signal of the data to be acquired based on the first original data to obtain second original data;
and the output module is used for comparing the acquisition algorithm parameters with the second original data and outputting to obtain compressed acquisition data.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the data acquisition method according to any one of claims 1 to 7 are implemented when the program is executed by the processor.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the data acquisition method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911271726.8A CN111010191B (en) | 2019-12-12 | 2019-12-12 | Data acquisition method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911271726.8A CN111010191B (en) | 2019-12-12 | 2019-12-12 | Data acquisition method, system, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111010191A true CN111010191A (en) | 2020-04-14 |
| CN111010191B CN111010191B (en) | 2023-11-07 |
Family
ID=70114279
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911271726.8A Active CN111010191B (en) | 2019-12-12 | 2019-12-12 | Data acquisition method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111010191B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111751658A (en) * | 2020-06-24 | 2020-10-09 | 国家电网有限公司大数据中心 | Signal processing method and device |
| CN111967154A (en) * | 2020-08-11 | 2020-11-20 | 贵州医科大学 | Substance component analysis method and system |
| CN113985474A (en) * | 2021-10-28 | 2022-01-28 | 北京石油化工学院 | Multi-node cooperative sensing method for micro-seismic monitoring based on Internet of things |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102938649A (en) * | 2012-09-27 | 2013-02-20 | 江苏大学 | Self-adaptive reconstruction and uncompressing method for power quality data based on compressive sensing theory |
| CN103346798A (en) * | 2013-06-05 | 2013-10-09 | 中国科学院微电子研究所 | Signal collecting method with sampling frequency lower than Nyquist frequency |
| CN103823133A (en) * | 2013-12-07 | 2014-05-28 | 西南交通大学 | On-line power quality monitoring system based on compression sensing |
| CN105181122A (en) * | 2015-08-06 | 2015-12-23 | 兰州理工大学 | Mechanical vibration signal data compression and acquisition method |
| CN106227705A (en) * | 2016-09-20 | 2016-12-14 | 北京邮电大学 | A kind of method and device of data collection |
| US20170188039A1 (en) * | 2015-12-23 | 2017-06-29 | Sony Corporation | Video encoding and decoding apparatus, system and method |
| CN108924148A (en) * | 2018-07-18 | 2018-11-30 | 中南大学 | A kind of source signal collaborative compression perception data restoration methods |
| CN109905129A (en) * | 2019-03-06 | 2019-06-18 | 湖南大学 | Low overhead electric power data acquisition method based on distributed compression sensing |
-
2019
- 2019-12-12 CN CN201911271726.8A patent/CN111010191B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102938649A (en) * | 2012-09-27 | 2013-02-20 | 江苏大学 | Self-adaptive reconstruction and uncompressing method for power quality data based on compressive sensing theory |
| CN103346798A (en) * | 2013-06-05 | 2013-10-09 | 中国科学院微电子研究所 | Signal collecting method with sampling frequency lower than Nyquist frequency |
| CN103823133A (en) * | 2013-12-07 | 2014-05-28 | 西南交通大学 | On-line power quality monitoring system based on compression sensing |
| CN105181122A (en) * | 2015-08-06 | 2015-12-23 | 兰州理工大学 | Mechanical vibration signal data compression and acquisition method |
| US20170188039A1 (en) * | 2015-12-23 | 2017-06-29 | Sony Corporation | Video encoding and decoding apparatus, system and method |
| CN106227705A (en) * | 2016-09-20 | 2016-12-14 | 北京邮电大学 | A kind of method and device of data collection |
| CN108924148A (en) * | 2018-07-18 | 2018-11-30 | 中南大学 | A kind of source signal collaborative compression perception data restoration methods |
| CN109905129A (en) * | 2019-03-06 | 2019-06-18 | 湖南大学 | Low overhead electric power data acquisition method based on distributed compression sensing |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111751658A (en) * | 2020-06-24 | 2020-10-09 | 国家电网有限公司大数据中心 | Signal processing method and device |
| CN111967154A (en) * | 2020-08-11 | 2020-11-20 | 贵州医科大学 | Substance component analysis method and system |
| CN113985474A (en) * | 2021-10-28 | 2022-01-28 | 北京石油化工学院 | Multi-node cooperative sensing method for micro-seismic monitoring based on Internet of things |
| CN113985474B (en) * | 2021-10-28 | 2023-10-31 | 北京石油化工学院 | Microseism monitoring multi-node collaborative sensing method based on Internet of things |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111010191B (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111010191B (en) | Data acquisition method, system, equipment and storage medium | |
| CN103037212B (en) | The adaptive block compressed sensing method for encoding images of view-based access control model perception | |
| CN103871058A (en) | Compressed sampling matrix decomposition-based infrared small target detection method | |
| CN109756740B (en) | Half tensor image compression method and image recovery method based on optimal measurement matrix | |
| CN108233943B (en) | Compressed sensing method based on minimum correlation measurement matrix | |
| Lu et al. | A physics-constrained dictionary learning approach for compression of vibration signals | |
| CN114332500A (en) | Image processing model training method and device, computer equipment and storage medium | |
| Progonov | Performance of statistical stegdetectors in case of small number of stego images in training set | |
| CN113708771B (en) | Half tensor product compressed sensing method based on Style algorithm | |
| Huang et al. | Optimized measurement matrix for compressive sensing | |
| CN109582917B (en) | Signal denoising method and device based on SSA, terminal equipment and storage medium | |
| Lin et al. | Perceptual image quality assessment: recent progress and trends | |
| CN115546271A (en) | Visual analysis method, device, equipment and medium based on depth joint characterization | |
| CN112634391B (en) | Gray image depth reconstruction and fault diagnosis system based on compressed sensing | |
| CN115147283A (en) | Image reconstruction method, device, equipment and medium | |
| CN111898493B (en) | Object identification method based on binary quantitative three-dimensional feature descriptor | |
| CN108599773B (en) | Vibration signal data compression acquisition method based on deterministic measurement matrix | |
| EP3269042B1 (en) | Data reduction method and apparatus | |
| CN116338242B (en) | Vibrating wire data acquisition instrument and control method thereof | |
| Song et al. | Security measure for image steganography based on high dimensional KL divergence | |
| CN107766294A (en) | Method and device for recovering missing data | |
| CN117319655B (en) | Image compression processing method, system, device and medium | |
| KR101753521B1 (en) | The transmission image processing method using a Bezier curve | |
| CN106204501B (en) | A kind of compressed sensing restoration methods | |
| WO2019148309A1 (en) | Quick reconstruction method and system for infrared small target image based on structure information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |