EP4318321A1 - Method and device for processing data conforming to statistical distribution - Google Patents
Method and device for processing data conforming to statistical distribution Download PDFInfo
- Publication number
- EP4318321A1 EP4318321A1 EP22781616.2A EP22781616A EP4318321A1 EP 4318321 A1 EP4318321 A1 EP 4318321A1 EP 22781616 A EP22781616 A EP 22781616A EP 4318321 A1 EP4318321 A1 EP 4318321A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- data
- neural network
- generated
- original
- original data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 15
- 238000012545 processing Methods 0.000 title claims description 15
- 238000013528 artificial neural network Methods 0.000 claims abstract description 41
- 238000012549 training Methods 0.000 claims abstract description 15
- 238000003672 processing method Methods 0.000 claims abstract description 4
- 238000004002 angle-resolved photoelectron spectroscopy Methods 0.000 claims description 24
- 238000013527 convolutional neural network Methods 0.000 claims description 11
- 238000004611 spectroscopical analysis Methods 0.000 claims description 4
- 238000011156 evaluation Methods 0.000 abstract 1
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 4
- 238000002047 photoemission electron microscopy Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 230000007423 decrease Effects 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 2
- 238000013434 data augmentation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
Definitions
- the present disclosure relates to a data processing method, in particular, to a method and device of processing data conforming to statistical distribution.
- Spectroscopy such as angle-resolved photoelectron spectroscopy (ARPES) and momentum resolved photoemission electron microscopy (k-PEEM) is used for the research of microscopic structure, for example, electronic structure of a wide range of materials.
- ARPES angle-resolved photoelectron spectroscopy
- k-PEEM momentum resolved photoemission electron microscopy
- An example of the methods of reducing the noise of data is Gaussian smoothing, but it may cause data blurring that may cause damage on the essential information.
- the embodiments of the present disclosure is provided to reduce the data acquisition time and noise.
- a data processing method includes: training a neural network; receiving input data from an external source, and converting the input data by the trained neural network, wherein the training comprises: generating one or more generated data from an original data; converting, by the neural network, the generated data into an output data; estimating the output data based on the original data; and optimizing the neural network based on result of the estimation, wherein the original data and the generated data conform to a statistical distribution, and wherein the original data and the output data have signal-to-noise ratio higher than the generated data.
- the generating may include generating the generated data at random.
- the statistical distribution may include Poisson distribution.
- the input data and the original data may include spectroscopy data.
- the input data and the original data may include angle-resolved photoelectron spectroscopy (ARPES) count data.
- ARPES angle-resolved photoelectron spectroscopy
- the number of the generated data may be equal to or greater than two.
- the neural network may include a deep neural network.
- the neural network may include a deep convolutional neural network.
- the deep convolutional neural network may include equal to or less than 20 layers.
- the estimating may be performed using a loss function; and the loss function may include weighted sum of mean absolute error and multiscale structural similarity index.
- a data processing device includes: a processor converting input data, wherein the processor comprises: a receiver receiving an original data; a generator generating one or more generated data at random from the original data; a neural network converting the generated data into an output data; and an estimator estimating the output data based on the original data, wherein the original data and the generated data conform to a statistical distribution, wherein the original data and the output data have signal-to-noise ratio higher than the generated data, and wherein the neural network is optimized according to output of the estimator.
- the statistical distribution may include Poisson distribution.
- the input data and the original data may include angle-resolved photoelectron spectroscopy (ARPES) count data.
- ARPES angle-resolved photoelectron spectroscopy
- the number of the generated data may be equal to or greater than two.
- the neural network may include a deep convolutional neural network having 20 layers or less.
- the estimator may estimate using a loss function; and the loss function may include weighted sum of mean absolute error and multiscale structural similarity index.
- the embodiments of the present disclosure can reduce the data acquisition time with further reduced noise.
- FIG. 1 is a schematic block diagram of a data processing device according to an embodiment of the present disclosure
- FIG. 2 is a schematic block diagram of a processor of a data processing device according to an embodiment of the present disclosure
- FIG. 3 illustrates an example of original data and generated data according to an embodiment of the present disclosure
- FIG. 4 schematically illustrates a process of training a neural network according to an embodiment of the present disclosure.
- a data processing device includes the processor 1 that processes input data IN to generate output data OUT.
- the input data IN may include spectroscopy data, for example, the data measured using angle-resolved photoelectron spectroscopy (ARPES), momentum resolved photoemission electron microscopy (k-PEEM), etc.
- An example of the input data IN may include ARPES count data.
- the processor 1 may convert the input data IN to produce the output data OUT, which has signal-to-noise ratio (SNR) higher than that of the input data IN.
- SNR signal-to-noise ratio
- the output data OUT from the processor 1 may be similar to those higher than the input data IN.
- the processor 1 may include a neural network 10, a receiver 20, a generator 30, and an estimator 40.
- the neural network 10 may include, for example, deep neural network or deep convolutional neural network, and may be trained using training data.
- the training data may include a plurality of data pairs of original data Di and generated data Dg.
- the receiver 20 is connected to the generator 30 and the estimator 40, and may receive the original data Di from an external source such as ARPES electronic analyzer (not shown), and transmit the original data Di to the generator 30 and the estimator 40.
- the generator 30, connected to the receiver 20 and the neural network 10, may generate the generated data Dg from the original data Di received from the receiver 20 using the statistical distribution to which the original data Di conforms and may transmit the generated data Dg to the neural network 10.
- the original data Di and the generated data Dg may conform to given statistical distribution such as Poisson distribution, but the embodiment is not limited thereto.
- the generated data Dg may be randomly generated from the original data Di by the generator 30, and two or more generated data Dg may be generated from one original datum Di.
- the generation of the generated data Dg may reduce overfitting of the neural network 10 during the training.
- the data set may be randomly rotated or flipped.
- the generated data Dg has a lower SNR than the original data Di.
- the original data Di may be ARPES high count data while the generated data Dg may be low count data.
- FIG. 3 shows original data, i.e., ARPES high count data
- (b) to (d) show examples of low count data generated from the original data in (a), where the counts of (b), (c) and (d) are 10 4 , 10 5 , and 10 6 , respectively. It is understood that the generated data becomes closer to the original count and the noise decreases more as the count becomes higher.
- the neural network 10 may convert an input, i.e., the generated data Dg to generate output data Do.
- the output data Do may have higher SNR than the generated data Dg.
- the estimator 40 may estimate the quality of the output data Do with reference to the original data Di.
- the estimation of the quality of the output data Do may use loss function or cost function, and the loss function may be defined, for example, as weighted sum of mean absolute error and multiscale structural similarity index.
- the estimation result may be fed back to the neural network 10.
- the neural network 10 may perform optimization, for example, adjusting inner parameters, and may repeat the generation of the output data Do, thereby proceeding the training.
- An example of optimization may include Adam optimization.
- the intensity of data may be randomly adjusted, and overfitting may be carefully checked using a validation data set which is not used in the training.
- FIGs. 5-8 illustrate low count data (a), output data (b), and high count data (c) according to embodiments of the present disclosure, which are energy (in FIGs. 5-7 ) or intensity (in FIG. 8 ) as function of momentum
- FIGs. 9 and 10 are graphs illustrating peak locations and widths obtained by line shape analysis of the data shown in FIGs, 7 and 8 , respectively
- FIG. 11 is a graph illustrating loss as function of the depth of the convolutional neural network.
- a deep network with 20 convolution layers was used in these experiments.
- Each layer of the convolutional neural network has a filter number of 64 and a kernel size of 3.
- An Adam optimizer was adopted to train the network for 150 epochs.
- the learning rate was initially set to be 5 ⁇ 10 -4 and multiplied by 0.1 after every 50 epochs.
- 50 different original high-count ARPES data were used, and 50 low-count data for each original data were randomly generated, resulting in a total of 2500 low-count data.
- ARPES data of FeSe along the M- ⁇ -M cut for 0.5 min (a) and 50 min (c) were obtained.
- the data (a) acquired for 0.5 min was input to the neural network to obtain output data (b), which in turn was compared with the data (c) acquired for 50 min.
- the data acquired for 0.5 min shows a high level of noise due to the low total counts.
- the output data (b) generated and denoised from the data (a) by the neural network is very similar to the data (c) acquired for 50 min.
- 0.5 min data (a) and 50 min data (c) were obtained, and output data (b) was obtained by inputting the 0.5 min data (a) to the neural network and compared with the 50 min data (c).
- the 0.5 min data (a) is barely visible, but the output data (b) processed by the neural network shows very clear shape with nearly no noise like the 50 min data (c).
- Output data (b) was obtained by inputting the 0.5 min data (a) to the neural network and compared with the 50 min data (c). As shown in FIG. 7 , the output data (b) has a very high SNR, which is very similar to the 50 min data (c), compared with highly-noisy 0.5 min data (a). The improvement of the SNR is more dominant in momentum distribution curve (MDC) shown in FIG. 8 .
- MDC momentum distribution curve
- the reason why the data processing, for example, ARPES data processing is effective for ARPES data as described above may be the correlation of the data among the neighboring pixels.
- Second, the length scale over which the band structure changes is larger than the data pixel size. Hence, the band structure does not change abruptly in a pixel. This means that the band structure has an approximate translational symmetry in a short length scale.
- the data processing according to embodiments of the present disclosure is more effective for the higher-dimensional data than for the lower dimensional data.
- the data processing according to embodiments of the present disclosure is more useful in higher dimension because the data acquisition time for higher-dimensional data is longer.
- the information loss decreases, for example, monotonically decreases with increasing depth, i.e., number of layers of the convolutional neural network.
- the network deeper than 20 layers could not be stably trained due to gradient vanishing/exploding.
- the embodiments of the present disclosure can reduce the data acquisition time with further reduced noise.
- the embodiments of the present disclosure can reduce the data acquisition time with further reduced noise.
Abstract
A data processing method according to an embodiment of the present invention comprises the steps of: training a neural network; receiving input data from the outside; and converting the received input data by means of the trained neural network, wherein the training step comprises the steps of: generating one or more pieces of generative data from raw data; converting the generative data into output data by means of the neural network; evaluating the output data on the basis of the raw data; and optimizing the neural network on the basis of the evaluation result, wherein the raw data and the generative data conform to a statistical distribution, and the raw data and the output data have higher signal-to-noise ratios than the generative data.
Description
- The present disclosure relates to a data processing method, in particular, to a method and device of processing data conforming to statistical distribution.
- Spectroscopy such as angle-resolved photoelectron spectroscopy (ARPES) and momentum resolved photoemission electron microscopy (k-PEEM) is used for the research of microscopic structure, for example, electronic structure of a wide range of materials. As the technology advances, its coverage increases to two and three dimensions. Accordingly, the time for acquiring data with given signal-to-noise ratio (SNR) may increase and insufficient data acquisition time may increase noise.
- An example of the methods of reducing the noise of data is Gaussian smoothing, but it may cause data blurring that may cause damage on the essential information.
- The embodiments of the present disclosure is provided to reduce the data acquisition time and noise.
- A data processing method according to an embodiment of the present disclosure includes: training a neural network; receiving input data from an external source, and converting the input data by the trained neural network, wherein the training comprises: generating one or more generated data from an original data; converting, by the neural network, the generated data into an output data; estimating the output data based on the original data; and optimizing the neural network based on result of the estimation, wherein the original data and the generated data conform to a statistical distribution, and wherein the original data and the output data have signal-to-noise ratio higher than the generated data.
- The generating may include generating the generated data at random.
- The statistical distribution may include Poisson distribution.
- The input data and the original data may include spectroscopy data.
- The input data and the original data may include angle-resolved photoelectron spectroscopy (ARPES) count data.
- The number of the generated data may be equal to or greater than two.
- The neural network may include a deep neural network.
- The neural network may include a deep convolutional neural network.
- The deep convolutional neural network may include equal to or less than 20 layers.
- The estimating may be performed using a loss function; and the loss function may include weighted sum of mean absolute error and multiscale structural similarity index.
- A data processing device according to an embodiment of the present disclosure includes: a processor converting input data, wherein the processor comprises: a receiver receiving an original data; a generator generating one or more generated data at random from the original data; a neural network converting the generated data into an output data; and an estimator estimating the output data based on the original data, wherein the original data and the generated data conform to a statistical distribution, wherein the original data and the output data have signal-to-noise ratio higher than the generated data, and wherein the neural network is optimized according to output of the estimator.
- The statistical distribution may include Poisson distribution.
- The input data and the original data may include angle-resolved photoelectron spectroscopy (ARPES) count data.
- The number of the generated data may be equal to or greater than two.
- The neural network may include a deep convolutional neural network having 20 layers or less.
- The estimator may estimate using a loss function; and the loss function may include weighted sum of mean absolute error and multiscale structural similarity index.
- The embodiments of the present disclosure can reduce the data acquisition time with further reduced noise.
-
-
FIG. 1 is a schematic block diagram of a data processing device according to an embodiment of the present disclosure. -
FIG. 2 is a schematic block diagram of a processor of a data processing device according to an embodiment of the present disclosure. -
FIG. 3 illustrates an example of original data and generated data according to an embodiment of the present disclosure. -
FIG. 4 schematically illustrates a process of training a neural network according to an embodiment of the present disclosure. -
FIGs. 5-8 illustrates low count data (a), output data (b), and high count data (c) according to embodiments of the present disclosure, which are energy (inFIGs. 5-7 ) or intensity (inFIG. 8 ) as function of momentum. -
FIGs. 9 and10 are graphs illustrating peak locations and widths obtained by line shape analysis of the data shown inFIGs, 7 and8 , respectively. -
FIG. 11 is a graph illustrating loss as function of the depth of the convolutional neural network. - The embodiments of the present disclosure will be described in detail with reference to accompanying drawings such that those skill in the art can easily implement the present invention. However, the embodiments of the present invention may not be limited thereto and achieved in various forms.
-
FIG. 1 is a schematic block diagram of a data processing device according to an embodiment of the present disclosure,FIG. 2 is a schematic block diagram of a processor of a data processing device according to an embodiment of the present disclosure,FIG. 3 illustrates an example of original data and generated data according to an embodiment of the present disclosure, andFIG. 4 schematically illustrates a process of training a neural network according to an embodiment of the present disclosure. - Referring to
FIG. 1 , a data processing device according to an embodiment of the present disclosure includes theprocessor 1 that processes input data IN to generate output data OUT. - The input data IN may include spectroscopy data, for example, the data measured using angle-resolved photoelectron spectroscopy (ARPES), momentum resolved photoemission electron microscopy (k-PEEM), etc. An example of the input data IN may include ARPES count data.
- The
processor 1 may convert the input data IN to produce the output data OUT, which has signal-to-noise ratio (SNR) higher than that of the input data IN. For example, when the input data IN is count data, the output data OUT from theprocessor 1 may be similar to those higher than the input data IN. - Referring to
FIG. 2 , theprocessor 1 may include aneural network 10, areceiver 20, agenerator 30, and anestimator 40. - The
neural network 10 may include, for example, deep neural network or deep convolutional neural network, and may be trained using training data. - The training data may include a plurality of data pairs of original data Di and generated data Dg.
thereceiver 20 is connected to thegenerator 30 and theestimator 40, and may receive the original data Di from an external source such as ARPES electronic analyzer (not shown), and transmit the original data Di to thegenerator 30 and theestimator 40. - The
generator 30, connected to thereceiver 20 and theneural network 10, may generate the generated data Dg from the original data Di received from thereceiver 20 using the statistical distribution to which the original data Di conforms and may transmit the generated data Dg to theneural network 10. The original data Di and the generated data Dg may conform to given statistical distribution such as Poisson distribution, but the embodiment is not limited thereto. The generated data Dg may be randomly generated from the original data Di by thegenerator 30, and two or more generated data Dg may be generated from one original datum Di. The generation of the generated data Dg may reduce overfitting of theneural network 10 during the training. For data augmentation, the data set may be randomly rotated or flipped. - According to an embodiment of the present disclosure, the generated data Dg has a lower SNR than the original data Di. For example, the original data Di may be ARPES high count data while the generated data Dg may be low count data. In
FIG. 3, (a) shows original data, i.e., ARPES high count data, and (b) to (d) show examples of low count data generated from the original data in (a), where the counts of (b), (c) and (d) are 104, 105, and 106, respectively. It is understood that the generated data becomes closer to the original count and the noise decreases more as the count becomes higher. - Referring to
FIG. 2 again, theneural network 10 may convert an input, i.e., the generated data Dg to generate output data Do. The output data Do may have higher SNR than the generated data Dg. - The
estimator 40 may estimate the quality of the output data Do with reference to the original data Di. The estimation of the quality of the output data Do may use loss function or cost function, and the loss function may be defined, for example, as weighted sum of mean absolute error and multiscale structural similarity index. The estimation result may be fed back to theneural network 10. Theneural network 10 may perform optimization, for example, adjusting inner parameters, and may repeat the generation of the output data Do, thereby proceeding the training. An example of optimization may include Adam optimization. - During the training, the intensity of data may be randomly adjusted, and overfitting may be carefully checked using a validation data set which is not used in the training.
- Some experimental examples of a data processing device according to embodiments of the present disclosure will be described in detail with reference to
FIGs. 5-11 . -
FIGs. 5-8 illustrate low count data (a), output data (b), and high count data (c) according to embodiments of the present disclosure, which are energy (inFIGs. 5-7 ) or intensity (inFIG. 8 ) as function of momentum,FIGs. 9 and10 are graphs illustrating peak locations and widths obtained by line shape analysis of the data shown inFIGs, 7 and8 , respectively, andFIG. 11 is a graph illustrating loss as function of the depth of the convolutional neural network. - A deep network with 20 convolution layers was used in these experiments. Each layer of the convolutional neural network has a filter number of 64 and a kernel size of 3. An Adam optimizer was adopted to train the network for 150 epochs. The learning rate was initially set to be 5 × 10-4 and multiplied by 0.1 after every 50 epochs. For the training data set, 50 different original high-count ARPES data were used, and 50 low-count data for each original data were randomly generated, resulting in a total of 2500 low-count data.
- Referring to
FIG. 5 , ARPES data of FeSe along the M-Γ-M cut for 0.5 min (a) and 50 min (c) were obtained. The data (a) acquired for 0.5 min was input to the neural network to obtain output data (b), which in turn was compared with the data (c) acquired for 50 min. The data acquired for 0.5 min shows a high level of noise due to the low total counts. As shown inFIG. 5 , although the data (a) acquired for 0.5 min shows a high level of noise, the output data (b) generated and denoised from the data (a) by the neural network is very similar to the data (c) acquired for 50 min. - Referring to
FIG. 6 , for the second derivative of the data shown inFIG. 5 , 0.5 min data (a) and 50 min data (c) were obtained, and output data (b) was obtained by inputting the 0.5 min data (a) to the neural network and compared with the 50 min data (c). As shown inFIG. 6 , the 0.5 min data (a) is barely visible, but the output data (b) processed by the neural network shows very clear shape with nearly no noise like the 50 min data (c). - Referring to
FIGs. 7 and8 , 0.5 min ARPES data (a) and 50 min ARPES data (c) of Bi-2212 along the nodal cut were obtained. Output data (b) was obtained by inputting the 0.5 min data (a) to the neural network and compared with the 50 min data (c). As shown inFIG. 7 , the output data (b) has a very high SNR, which is very similar to the 50 min data (c), compared with highly-noisy 0.5 min data (a). The improvement of the SNR is more dominant in momentum distribution curve (MDC) shown inFIG. 8 . Referring toFIGs. 9 and10 , in the graph of peak location and width obtained by performing line shape analysis fitting MDC, 0.5 min data processed by the neural network is almost identical to 50 min data as shown in (b) ofFIGs. 9 and10 , demonstrating that the data processing preserves the quantitative information of the band structure. - The reason why the data processing, for example, ARPES data processing is effective for ARPES data as described above may be the correlation of the data among the neighboring pixels. Two major factors contribute to the correlation. First, the dimension for ARPES features is larger than the pixel size, leading to occupation of several pixels for an ARPES feature. Thus, if the value at a pixel is large, it is likely for neighboring pixels to have a large value. Second, the length scale over which the band structure changes is larger than the data pixel size. Hence, the band structure does not change abruptly in a pixel. This means that the band structure has an approximate translational symmetry in a short length scale. Even if the information at a pixel is corrupted with noise, the value at the pixel can be recovered from the most statistically probable value inferred from adjacent pixel values. Therefore, a data set carries more information than just the pixel-wise sum of information. With the additional information, the seemingly imperfect information of the noisy data can be recovered.
- As such, since the information at a pixel is compensated by the information from adjacent pixel values, better performance is expected if there are more neighboring pixels. Therefore, the data processing according to embodiments of the present disclosure is more effective for the higher-dimensional data than for the lower dimensional data. The data processing according to embodiments of the present disclosure is more useful in higher dimension because the data acquisition time for higher-dimensional data is longer.
- Referring to
FIG. 11 , it is shown that the information loss decreases, for example, monotonically decreases with increasing depth, i.e., number of layers of the convolutional neural network. The network deeper than 20 layers could not be stably trained due to gradient vanishing/exploding. - The embodiments of the present disclosure can reduce the data acquisition time with further reduced noise.
- The embodiments of the present disclosure can reduce the data acquisition time with further reduced noise.
Claims (16)
- A data processing method comprising:training a neural network;receiving input data from an external source, andconverting the input data by the trained neural network,wherein the training comprises:generating one or more generated data from an original data;converting, by the neural network, the generated data into an output data;estimating the output data based on the original data; andoptimizing the neural network based on result of the estimation,wherein the original data and the generated data conform to a statistical distribution, andwherein the original data and the output data have signal-to-noise ratio higher than the generated data.
- The method of claim 1, wherein the generating comprises generating the generated data at random.
- The method of claim 1, wherein the statistical distribution comprises Poisson distribution.
- The method of claim 1, wherein the input data and the original data comprises spectroscopy data.
- The method of claim 4, wherein the input data and the original data comprises angle-resolved photoelectron spectroscopy (ARPES) count data.
- The method of claim 1, wherein the number of the generated data is equal to or greater than two.
- The method of claim 1, wherein the neural network comprises a deep neural network.
- The method of claim 1, wherein the neural network comprises a deep convolutional neural network.
- The method of claim 8, wherein the deep convolutional neural network comprises equal to or less than 20 layers.
- The method of claim 1, wherein:the estimating is performed using a loss function; andthe loss function comprises weighted sum of mean absolute error and multiscale structural similarity index.
- A data processing device comprising:a processor converting input data,wherein the processor comprises:a receiver receiving an original data;a generator generating one or more generated data at random from the original data;a neural network converting the generated data into an output data; andan estimator estimating the output data based on the original data,wherein the original data and the generated data conform to a statistical distribution,wherein the original data and the output data have signal-to-noise ratio higher than the generated data, andwherein the neural network is optimized according to output of the estimator.
- The device of claim 11, wherein the statistical distribution comprises Poisson distribution.
- The device of claim 11, wherein the input data and the original data comprises angle-resolved photoelectron spectroscopy (ARPES) count data.
- The device of claim 11, wherein the number of the generated data is equal to or greater than two.
- The device of claim 11, wherein the neural network comprises a deep convolutional neural network having 20 layers or less.
- The device of claim 11, wherein:the estimator estimates using a loss function; andthe loss function comprises weighted sum of mean absolute error and multiscale structural similarity index.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020210041786A KR20220135724A (en) | 2021-03-31 | 2021-03-31 | Method and device of processing data conforming to statistical distribution |
PCT/KR2022/004518 WO2022211497A1 (en) | 2021-03-31 | 2022-03-30 | Method and device for processing data conforming to statistical distribution |
Publications (1)
Publication Number | Publication Date |
---|---|
EP4318321A1 true EP4318321A1 (en) | 2024-02-07 |
Family
ID=83459504
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP22781616.2A Pending EP4318321A1 (en) | 2021-03-31 | 2022-03-30 | Method and device for processing data conforming to statistical distribution |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP4318321A1 (en) |
KR (1) | KR20220135724A (en) |
WO (1) | WO2022211497A1 (en) |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9225889B1 (en) * | 2014-08-18 | 2015-12-29 | Entropix, Inc. | Photographic image acquisition device and method |
EP3278559B1 (en) * | 2015-03-31 | 2021-05-05 | Magic Pony Technology Limited | Training end-to-end video processes |
US10007977B2 (en) * | 2015-05-11 | 2018-06-26 | Netflix, Inc. | Techniques for predicting perceptual video quality |
-
2021
- 2021-03-31 KR KR1020210041786A patent/KR20220135724A/en not_active Application Discontinuation
-
2022
- 2022-03-30 EP EP22781616.2A patent/EP4318321A1/en active Pending
- 2022-03-30 WO PCT/KR2022/004518 patent/WO2022211497A1/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
KR20220135724A (en) | 2022-10-07 |
WO2022211497A1 (en) | 2022-10-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN109035149B (en) | License plate image motion blur removing method based on deep learning | |
Chaudhury et al. | Non-local Euclidean medians | |
Hoshyar et al. | Comparing the performance of various filters on skin cancer images | |
Mukhopadhyay et al. | Wavelet based denoising of medical images using sub-band adaptive thresholding through genetic algorithm | |
CN112396110A (en) | Method for generating anti-cascade network augmented image | |
Tang et al. | Noise estimation of natural images via statistical analysis and noise injection | |
CN112288714B (en) | Hardware Trojan horse detection method based on deep learning | |
CN112991199B (en) | Image high-low frequency decomposition noise removal method based on residual dense network | |
CN114331886A (en) | Image deblurring method based on depth features | |
CN115097398A (en) | Radar anti-interference signal recovery method based on cross-domain signal low-loss recovery network | |
CN116563146A (en) | Image enhancement method and system based on leachable curvature map | |
EP4318321A1 (en) | Method and device for processing data conforming to statistical distribution | |
Abdulah et al. | Review Study of Image De-Noising on Digital Image Processing and Applications | |
Fursov | Identification of square-exponential FIR-filter parameters in the absence of a test image | |
CN111695444A (en) | Radiation source individual feature extraction method based on wave atomic transformation | |
Chaudhary et al. | Simultaneous denoising and edge estimation from SEM images using deep convolutional neural networks | |
Shruthi et al. | Constrained least squares filtering followed by denoising of decomposed images using wave atom and wavelet transform | |
CN114881215A (en) | Millimeter wave radar data screening neural network processing method | |
Alawsi et al. | Performance Analysis of Noise Removal Techniques For Digital Images | |
CN114627340B (en) | Image steganography detection feature self-adaptive selection method based on triple measurement | |
Charmouti et al. | Progression approach for image denoising | |
Sang et al. | Suppressing seismic coherent and incoherent noise via deep neural network | |
Bhuiyan et al. | Edge detection via a fast and adaptive bidimensional empirical mode decomposition | |
Xu et al. | Adaptive image denoising in mixed gaussian-impulse noise using weighted nuclear norm minimization | |
Sun et al. | Adequate determination of a band of wavelet threshold for noise cancellation using particle swarm optimization |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20231030 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |