CN111159847A - Method and system for automatically fitting small-angle scattering data - Google Patents
Method and system for automatically fitting small-angle scattering data Download PDFInfo
- Publication number
- CN111159847A CN111159847A CN201911223151.2A CN201911223151A CN111159847A CN 111159847 A CN111159847 A CN 111159847A CN 201911223151 A CN201911223151 A CN 201911223151A CN 111159847 A CN111159847 A CN 111159847A
- Authority
- CN
- China
- Prior art keywords
- point
- output
- loss function
- fitted
- gradient descent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Processing (AREA)
Abstract
The invention provides a method and a system for automatically fitting small-angle scattering data, which comprises the following steps: a data acquisition step: acquiring input small-angle scattering experimental data, parameters to be fitted and value ranges of the parameters to be fitted, wherein the value ranges of all the parameters to be fitted form a search space together; heuristic algorithm processing steps: inputting the search space into a heuristic algorithm to obtain a first output; gradient descent method treatment steps: taking the first output as the input of a gradient descent method to obtain a second output; grid search method processing steps: and taking the second output as the input of the grid search method to obtain the final fitting result. The invention realizes the automatic fitting of the small-angle scattering data by combining a heuristic algorithm, a gradient descent method and a grid search method. Therefore, the influence caused by human subjectivity in the data processing process is greatly reduced, and the accuracy, stability and high efficiency of data processing are improved.
Description
Technical Field
The invention relates to the field of data processing, in particular to a method and a system for automatically fitting small-angle scattering data.
Background
In the field of small-angle scattering, all data processing methods or software need to be manually participated and operated, so that the result of data processing is inevitably influenced by human subjectivity, and even different data processors can obtain different results for the same data.
For the existing small-angle scattering data processing method or software, patent CN 107589136a proposes a double-model fitting method for small-angle X-ray scattering, and in addition, the JOURNAL "joint application digital crystal log phy" in 2015 8 months, page 1587 discloses "SASfit software", the JOURNAL "joint application digital crystal log phy" in 2017 months, page 1212 discloses "atas software", which can both fit small-angle scattering data well, but they do not have a complete automated fitting process.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a method and a system for automatically fitting small-angle scattering data.
The method for automatically fitting the small-angle scattering data provided by the invention comprises the following steps:
a data acquisition step: acquiring input small-angle scattering experimental data, parameters to be fitted and value ranges of the parameters to be fitted, wherein the value ranges of all the parameters to be fitted form a search space together;
heuristic algorithm processing steps: inputting the search space into a heuristic algorithm to obtain a first output;
gradient descent method treatment steps: taking the first output as the input of a gradient descent method to obtain a second output;
grid search method processing steps: and taking the second output as the input of the grid search method to obtain the final fitting result.
Preferably, a set of values of all the parameters to be fitted is a point in the search space, each point in the search space corresponds to a fitting curve, and there is a unique true solution corresponding to the small-angle scattering experimental data.
Preferably, the heuristic algorithmic processing step comprises:
randomly selecting at least one point in the search space, calculating the distance from the corresponding fitted curve to a real solution through a loss function f1, taking the selected point as a search history, reselecting at least one point according to the search history, calculating the distance from the corresponding fitted curve to the real solution through a loss function f1, taking the selected point as a new search history, and repeating the steps;
selecting a point closest to a true solution from the most recent search history as the first output.
Preferably, the gradient descent method processing step includes:
determining a new loss function f2, using the first output as the initial point of the loss function f 2;
the objective of the gradient descent method is to minimize the loss function f2, and from the initial point, at each iteration, the gradient of the loss function f2 at the current point is calculated and advances in the direction of the fastest gradient descent until a local optimal solution is reached as the second output.
Preferably, the grid search processing step includes:
generating a grid according to the second output and the accuracy requirement of each parameter to be fitted, determining a new loss function f3, calculating the distance between each point in the grid and a real solution by using the loss function f3, and selecting the point with the closest distance;
and if the selected point is the same as the second output, taking the selected point as a final fitting result, otherwise, taking the selected point as an initial point of the loss function f2, and repeating the gradient descent method processing step and the grid search method processing step.
According to the invention, the system for automatically fitting the small-angle scattering data comprises:
a data acquisition module: acquiring input small-angle scattering experimental data, parameters to be fitted and value ranges of the parameters to be fitted, wherein the value ranges of all the parameters to be fitted form a search space together;
a heuristic algorithm processing module: inputting the search space into a heuristic algorithm to obtain a first output;
gradient descent method processing module: taking the first output as the input of a gradient descent method to obtain a second output;
the grid search method processing module: and taking the second output as the input of the grid search method to obtain the final fitting result.
Preferably, a set of values of all the parameters to be fitted is a point in the search space, each point in the search space corresponds to a fitting curve, and there is a unique true solution corresponding to the small-angle scattering experimental data.
Preferably, the heuristic algorithm processing module comprises:
randomly selecting at least one point in the search space, calculating the distance from the corresponding fitted curve to a real solution through a loss function f1, taking the selected point as a search history, reselecting at least one point according to the search history, calculating the distance from the corresponding fitted curve to the real solution through a loss function f1, taking the selected point as a new search history, and repeating the steps;
selecting a point closest to a true solution from the most recent search history as the first output.
Preferably, the gradient descent method processing module includes:
determining a new loss function f2, using the first output as the initial point of the loss function f 2;
the objective of the gradient descent method is to minimize the loss function f2, and from the initial point, at each iteration, the gradient of the loss function f2 at the current point is calculated and advances in the direction of the fastest gradient descent until a local optimal solution is reached as the second output.
Preferably, the grid searching method processing module includes:
generating a grid according to the second output and the accuracy requirement of each parameter to be fitted, determining a new loss function f3, calculating the distance between each point in the grid and a real solution by using the loss function f3, and selecting the point with the closest distance;
if the selected point is the same as the second output, the final fitting result is obtained, otherwise, the selected point is used as the initial point of the loss function f2, and the gradient descent method processing and the grid search method processing are repeated.
Compared with the prior art, the invention has the following beneficial effects:
the invention realizes the automatic fitting of the small-angle scattering data by combining a heuristic algorithm, a gradient descent method and a grid search method. By automatically fitting the small-angle scattering data, the influence caused by human subjectivity in the data processing process is greatly reduced, and meanwhile, the accuracy, stability and efficiency of data processing are improved.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:
FIG. 1 is a flow chart of the operation of the present invention.
Detailed Description
The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that it would be obvious to those skilled in the art that various changes and modifications can be made without departing from the spirit of the invention. All falling within the scope of the present invention.
As shown in fig. 1, the present invention provides a method for automatically fitting small-angle scattering data, including:
a data acquisition step: the method comprises the steps of obtaining input small-angle scattering experimental data, parameters to be fitted and value ranges of the parameters to be fitted, wherein the value ranges of all the parameters to be fitted jointly form a search space. After the user inputs and sets the parameters, the value ranges of all the parameters to be fitted form a search space together; one group of values of all parameters to be fitted is called a point in a search space; and (4) calculating by a formula, wherein each point in the search space corresponds to a fitting curve. The experimental data input by the user can be regarded as an experimental curve, and only one real solution corresponding to the experimental curve exists in the search space.
Heuristic algorithm processing steps: the search space is input into a heuristic algorithm to obtain a first output. Heuristic algorithms such as bayesian optimization, simulated annealing algorithms, particle swarm optimization algorithms, genetic algorithms, etc. Taking bayesian optimization as an example:
firstly, carrying out Bayesian optimization to randomly select a plurality of points in a search space, and calculating fitting curves of the points and distances between the fitting curves and an experimental curve; the distance is calculated by a specific loss function f1 (such as least square, chi-square statistic, least square + penalty term, etc.), and the distance can be used for measuring the distance between a point and a real solution; these points together constitute a search history; next, calculating the next or next batch of new points, the fitted curves of these new points and their distances from the experimental curve according to the search history; meanwhile, the new points and the search history form a new search history; this process is called a cycle; after a number of cycles, the point closest to the true solution is selected from the latest search history as the first output.
Gradient descent method treatment steps: and taking the first output as the input of the gradient descent method to obtain a second output. Firstly, determining a new loss function f2 as the loss function of the gradient descent method; the input (first output) of the gradient descent method is set as the initial point of f 2. The goal of the gradient descent method is to minimize f2, i.e., minimize the distance between the current point and the true solution; starting from the initial point, at each iteration, the gradient descent method calculates the gradient of f2 at the current point and advances in the direction of the fastest gradient descent until a local optimal solution is reached as the second output.
Grid search method processing steps: and taking the second output as the input of the grid search method to obtain the final fitting result. First, a grid is generated based on the second output and the accuracy requirements of each parameter to be fitted (which may be set by the user; otherwise, default values are used), and a new loss function f3 is determined (e.g., CorMap + chi-square statistic + penalty term, etc.). Then, calculating the distance between each point in the grid and the real solution by using f3, and selecting the point with the closest distance; if the point is the same as the second output, the point is taken as a third output, otherwise, the point is taken as an initial point of the loss function f2, and the gradient descent method and the grid search method are repeated.
Finally, the third output is taken as the final result of the whole method. The third output may be a result of characterization of the sample by small angle scattering, which indirectly reflects structural information of the nanostructures in the sample. For example, the third output includes the average size of the nanostructure, and compared with the size of the nanostructure directly observed by an electron microscope, the result given by the third output is more statistical and representative. In addition, the third output also contains 3-dimensional information of the nanostructure, such as volume fraction, etc. Because the third output is obtained through an automatic data fitting strategy, the method is particularly suitable for occasions with large data volume and needing real-time interpretation, such as experiment centers of various scientific large devices and the like.
In addition, the automatic data fitting method is only suitable for small-angle scattering data, and any other one-dimensional experimental data can be fitted by using the strategy.
On the basis of the method for automatically fitting the small-angle scattering data, the invention also provides a system for automatically fitting the small-angle scattering data, which comprises the following steps:
a data acquisition module: the method comprises the steps of obtaining input small-angle scattering experimental data, parameters to be fitted and value ranges of the parameters to be fitted, wherein the value ranges of all the parameters to be fitted jointly form a search space.
A heuristic algorithm processing module: the search space is input into a heuristic algorithm to obtain a first output.
Gradient descent method processing module: and taking the first output as the input of the gradient descent method to obtain a second output.
The grid search method processing module: and taking the second output as the input of the grid search method to obtain the final fitting result.
Those skilled in the art will appreciate that, in addition to implementing the system and its various devices, modules, units provided by the present invention as pure computer readable program code, the system and its various devices, modules, units provided by the present invention can be fully implemented by logically programming method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Therefore, the system and various devices, modules and units thereof provided by the invention can be regarded as a hardware component, and the devices, modules and units included in the system for realizing various functions can also be regarded as structures in the hardware component; means, modules, units for performing the various functions may also be regarded as structures within both software modules and hardware components for performing the method.
The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes or modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention. The embodiments and features of the embodiments of the present application may be combined with each other arbitrarily without conflict.
Claims (10)
1. A method of automatically fitting small angle scatter data, comprising:
a data acquisition step: acquiring input small-angle scattering experimental data, parameters to be fitted and value ranges of the parameters to be fitted, wherein the value ranges of all the parameters to be fitted form a search space together;
heuristic algorithm processing steps: inputting the search space into a heuristic algorithm to obtain a first output;
gradient descent method treatment steps: taking the first output as the input of a gradient descent method to obtain a second output;
grid search method processing steps: and taking the second output as the input of the grid search method to obtain the final fitting result.
2. The method of claim 1, wherein a set of values for all parameters to be fitted is a point in the search space, each point in the search space corresponds to a fitting curve, and there is a unique true solution corresponding to the small-angle scattering experimental data.
3. The method of automatically fitting small angle scatter data of claim 1, wherein the heuristic processing step comprises:
randomly selecting at least one point in the search space, calculating the distance from the corresponding fitted curve to a real solution through a loss function f1, taking the selected point as a search history, reselecting at least one point according to the search history, calculating the distance from the corresponding fitted curve to the real solution through a loss function f1, taking the selected point as a new search history, and repeating the steps;
selecting a point closest to a true solution from the most recent search history as the first output.
4. The method of automatically fitting small angle scatter data of claim 3, wherein the gradient descent method processing step comprises:
determining a new loss function f2, using the first output as the initial point of the loss function f 2;
the objective of the gradient descent method is to minimize the loss function f2, and from the initial point, at each iteration, the gradient of the loss function f2 at the current point is calculated and advances in the direction of the fastest gradient descent until a local optimal solution is reached as the second output.
5. The method of automatically fitting small angle scatter data of claim 4, wherein the grid search processing step comprises:
generating a grid according to the second output and the accuracy requirement of each parameter to be fitted, determining a new loss function f3, calculating the distance between each point in the grid and a real solution by using the loss function f3, and selecting the point with the closest distance;
and if the selected point is the same as the second output, taking the selected point as a final fitting result, otherwise, taking the selected point as an initial point of the loss function f2, and repeating the gradient descent method processing step and the grid search method processing step.
6. A system for automatically fitting small angle scatter data, comprising:
a data acquisition module: acquiring input small-angle scattering experimental data, parameters to be fitted and value ranges of the parameters to be fitted, wherein the value ranges of all the parameters to be fitted form a search space together;
a heuristic algorithm processing module: inputting the search space into a heuristic algorithm to obtain a first output;
gradient descent method processing module: taking the first output as the input of a gradient descent method to obtain a second output;
the grid search method processing module: and taking the second output as the input of the grid search method to obtain the final fitting result.
7. The system of claim 6, wherein a set of values for all parameters to be fitted is a point in the search space, each point in the search space corresponds to a fitting curve, and there is a unique true solution corresponding to the small-angle scattering experimental data.
8. The system for automatically fitting small angle scattering data of claim 6, wherein the heuristic processing module comprises:
randomly selecting at least one point in the search space, calculating the distance from the corresponding fitted curve to a real solution through a loss function f1, taking the selected point as a search history, reselecting at least one point according to the search history, calculating the distance from the corresponding fitted curve to the real solution through a loss function f1, taking the selected point as a new search history, and repeating the steps;
selecting a point closest to a true solution from the most recent search history as the first output.
9. The system for automatically fitting small angle scatter data of claim 8, wherein the gradient descent method processing module comprises:
determining a new loss function f2, using the first output as the initial point of the loss function f 2;
the objective of the gradient descent method is to minimize the loss function f2, and from the initial point, at each iteration, the gradient of the loss function f2 at the current point is calculated and advances in the direction of the fastest gradient descent until a local optimal solution is reached as the second output.
10. The system for automatically fitting small angle scattering data of claim 9, wherein the grid search processing module comprises:
generating a grid according to the second output and the accuracy requirement of each parameter to be fitted, determining a new loss function f3, calculating the distance between each point in the grid and a real solution by using the loss function f3, and selecting the point with the closest distance;
and if the selected point is the same as the second output, taking the selected point as a final fitting result, otherwise, taking the selected point as an initial point of the loss function f2, and repeating the gradient descent method processing step and the grid search method processing step.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911223151.2A CN111159847A (en) | 2019-12-03 | 2019-12-03 | Method and system for automatically fitting small-angle scattering data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911223151.2A CN111159847A (en) | 2019-12-03 | 2019-12-03 | Method and system for automatically fitting small-angle scattering data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111159847A true CN111159847A (en) | 2020-05-15 |
Family
ID=70556420
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911223151.2A Pending CN111159847A (en) | 2019-12-03 | 2019-12-03 | Method and system for automatically fitting small-angle scattering data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111159847A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103324939A (en) * | 2013-03-15 | 2013-09-25 | 江南大学 | Deviation classification and parameter optimization method based on least square support vector machine technology |
| CN104778338A (en) * | 2015-05-03 | 2015-07-15 | 长春工业大学 | Optimization method for set value of low-energy-consumption molecular distillation process |
| CN105785424A (en) * | 2016-02-25 | 2016-07-20 | 中国人民解放军63973部队 | Cadmium zinc telluride detector gamma spectrum full-energy peak non-linear fitting algorithm |
-
2019
- 2019-12-03 CN CN201911223151.2A patent/CN111159847A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103324939A (en) * | 2013-03-15 | 2013-09-25 | 江南大学 | Deviation classification and parameter optimization method based on least square support vector machine technology |
| CN104778338A (en) * | 2015-05-03 | 2015-07-15 | 长春工业大学 | Optimization method for set value of low-energy-consumption molecular distillation process |
| CN105785424A (en) * | 2016-02-25 | 2016-07-20 | 中国人民解放军63973部队 | Cadmium zinc telluride detector gamma spectrum full-energy peak non-linear fitting algorithm |
Non-Patent Citations (3)
| Title |
|---|
| 焦李成: "《计算智能导论》", 30 September 2019 * |
| 陈学武、程龙: "《城市低收入人群出行方式选择机理与交通改善策略》", 31 July 2018 * |
| 龚倩: "基于遗传算法和共轭梯度下降法的自适应振动光谱峰值拟合算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wang et al. | Differential evolution based on covariance matrix learning and bimodal distribution parameter setting | |
| US8948494B2 (en) | Unbiased wafer defect samples | |
| WO2016101628A1 (en) | Data processing method and device in data modeling | |
| CN107015875B (en) | Method and device for evaluating storage life of electronic complete machine | |
| Moreno et al. | Decomposition-based algorithms for the crew scheduling and routing problem in road restoration | |
| US9582869B2 (en) | Dynamic binning for diversification and defect discovery | |
| CN113205225A (en) | Method, system and data platform for identifying key factors of carbon emission peak | |
| Chowdhury et al. | Adaptive regulatory genes cardinality for reconstructing genetic networks | |
| CN111261235A (en) | Method and apparatus for retrieving structure of cyclic molecule and recording medium | |
| Ye et al. | Big data processing framework for manufacturing | |
| CN106648839A (en) | Method and device for processing data | |
| CN111159847A (en) | Method and system for automatically fitting small-angle scattering data | |
| CN112445690A (en) | Information acquisition method and device and electronic equipment | |
| CN115619200B (en) | Scheduling and multi-functional scheduling combination optimization method and device for split-type serum | |
| CN108958917A (en) | The method for scheduling task and system of cloud computing | |
| Yang et al. | On stochastic optimization and its applications to manufacturing | |
| Damgrave et al. | Rationalizing virtual reality based on manufacturing paradigms | |
| Cai et al. | Application research and improvement of particle swarm optimization algorithm | |
| JP2022079376A (en) | Optimal solution search device and optimal solution search program | |
| CN113780334A (en) | High-dimensional data classification method based on two-stage mixed feature selection | |
| CN113066523A (en) | Lepidoptera micro-nano structure unified characterization method and system based on space trigonometric function | |
| CN110751982B (en) | Intelligent parallelization knockout strategy screening method and system | |
| CN112926227A (en) | System reliability evaluation method and system based on subcomponent reliability function | |
| Ahmed et al. | A new method for solving deadlock using genetic algorithms | |
| CN110895542A (en) | High-risk SQL statement screening method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200515 |