JPH05266207A - Data smoothing device - Google Patents

Abstract

PURPOSE:To provide the data smoothing device which properly smooths data including unknown noise regardless of unknown classification and magnitude of the noise included in data as the object of smoothing. CONSTITUTION:A data smoothing device 1 is provided with a linearity evaluation value arithmetic part 11 which operates the linearity evaluation value of a dot string of inputted data, a linearity evaluation value discriminating part 12 which discriminates whether the linearity evaluation value obtained by this arithmetic part 11 is equal to or larger than a threshold or not, a smoothed line determining part 13 which determines a line obtained by flattening or smoothing input data based on the linearity evaluation value in the case of the linearity evaluation value equal to or larger than the threshold, and a parameter in minor-axis direction automatic change part 14 which corrects the value of a parameter corresponding to the length of the minor axis of an ellipse of a three-dimensional ellipse membership function in the case of the linearity evaluation value smaller than the threshold.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data smoothing device used for measuring a true value from a large number of data including noise and stochastic variations.

[0002]

2. Description of the Related Art As represented by the state observation of a system such as a plant, due to noise and stochastic variations, it is necessary to accurately grasp or measure the state of the system and the tendency of the state change from the observed value itself. There are many cases where you cannot do it. Even in such a case, if the mathematical model of the system can be accurately described and the statistical properties of noise are known, state estimation can be performed by the Kalman filter.

However, many systems that actually exist cannot be described by a mathematical model, and it is naturally difficult to perform objective state estimation such as the Kalman filter for these systems.

For example, for an observed value disturbed by non-stationary noise, the observed value itself cannot be treated as a system state or a tendency of state change. In this case, in the conventional technique, analysis is performed by a statistical method such as a regression line or a moving average. However, when exceptionally large noise (spike noise) is added, the conventional statistics are used. The statistical method cannot estimate the state with high reliability. Therefore, in order to remove such an exceptional noise, another method (for example, differentiation) has to be used.

As described above, the observed value of the actual system state includes non-stationary noise and spike noise,
According to the conventional data processing technology, in order to remove these noises, different methods must be used depending on the type of noise, and data with different types of noise mixed together can be processed appropriately by one method. I couldn't.

On the other hand, human beings can easily remove noise by subjective judgment, and can easily perform state estimation with relatively high reliability, that is, smoothing (smoothing) a data string to a true value. Can be done. This is the work of flattening the graph by ambiguous knowledge and judgment when analyzing the graph of observed data, and it does not necessarily perform accurate state estimation, but such data smoothing does not It is a very important and effective data processing technology for displaying observed values and performing predictive control.

Therefore, the inventor of the present invention automatically performs a smoothing process like that performed by humans on data including noise as described above, and thus data having noise or stochastic variation on the coordinate plane. When there is a sequence of points, the technique of evaluating the linearity of the sequence by fuzzy theory and using the method of finding the angle to reduce the components of noise and variations and output smooth lines of data. Was developed (Japanese Patent Application No. 2-243353).

This is because, for data represented by a plurality of points distributed on a coordinate plane, the center of an ellipse of a predetermined ellipse membership function is selected near the center of a point sequence selected from the plurality of points. Move to a point, rotate the ellipse about that point, sum of the goodness of fit by the ellipse membership function of each point at an arbitrary rotation angle θ, and the ellipse of the point projected on the major axis of the ellipse. The ratio of the goodness of fit by the membership function to the sum of the goodness of fit is taken as the linearity evaluation value E (θ) of the data point sequence, and the angle ψ of the major axis of the ellipse that maximizes this linearity evaluation value.
To measure. At the time of the first angle measurement, draw a line passing through the center of the ellipse at that time and having the angle to the middle of the center of the ellipse and the center of the ellipse at the next angle measurement,
After that, the operation of drawing a straight line from the end point of the line obtained by the previous angle measurement to the end point of the line obtained by the angle measurement at that time is sequentially repeated to obtain a line in which the data point sequence is smoothed. According to this technique, a line obtained by smoothing the data obtained by observation can be obtained, and data smoothing close to human's sensory judgment is realized.

[0009]

In the above data smoothing technique, the linearity evaluation value E (θ) depends on the parameter a _y that determines the fuzzy entropy in the minor axis direction of the three-dimensional elliptic membership function. To be done. But,
When the linearity evaluation value E becomes a small value (empirically, about 0.7 or less) due to a change in noise type or level (magnitude), the measured angle ψ becomes unsuitable, and smoothing is smoothed. There is a problem that you can not do well.

For example, when time-series data as shown in FIG. 8 is given as a target of smoothing, it is rare to smooth the data in the first half part (value of the horizontal axis is 0 to 1500) in the figure. Since only large spike noise is removed, the parameter a _{y in} the minor axis direction of the three-dimensional elliptic membership function is set to a relatively small value. Then, the smoothing result is as shown in FIG. That is, the spike noise is removed, but FIG.
In order to smooth the data in the latter half (values on the horizontal axis of 1600 to 2300) of the above, the set value of the parameter a _y is not appropriate for the type and level (magnitude) of noise, so smoothing has been performed. The line moves away from the actual data. Further, if the parameter a _y in the minor axis direction of the ellipse is set large assuming the case shown in the latter half of FIG. 8, spike noise may not be successfully removed in some cases.

Therefore, an object of the present invention is to perform data smoothing that can appropriately smooth data including unknown noise even if the type and magnitude of noise contained in the data to be smoothed is not known in advance. To provide a device.

[0012]

According to the present invention, when data represented by a plurality of points distributed on a coordinate plane is input, the center of an ellipse of a predetermined ellipse membership function is set to the plurality of points. Move to a point near the center of the point sequence selected from
The ellipse is rotated about that point, and the sum of the goodness of fit by the elliptic membership function of each point at any rotation angle θ and the elliptic membership function of the point projecting each point on the major axis of the ellipse The ratio with the sum of the goodness of fit is calculated, the ratio is used as the linearity evaluation value E (θ) of the data, and the angle ψ of the major axis of the ellipse that maximizes this value is measured
At the time of the first angle measurement, a line passing through the center of the ellipse at that time and having the angle ψ is drawn to near the center between the center of the ellipse and the center of the ellipse in the next angle measurement, and thereafter,
In the data smoothing device that outputs a line obtained by smoothing the data by sequentially repeating the operation of drawing a straight line from the end point of the line obtained by the previous angle measurement to the end point of the line obtained by the angle measurement at that time, By automatically adjusting the parameter a _y, which corresponds to the length of the minor axis of the ellipse of the membership function, according to the type and magnitude of noise, a line that is appropriately smoothed even for data containing unknown noise can be obtained. It is characterized by generating.

The data smoothing device having the above-mentioned function includes a linearity evaluation value calculation unit for calculating a linearity evaluation value for a point sequence of input data, and a straight line obtained by the linearity evaluation value calculation unit. A linearity evaluation value determination unit that determines whether the linearity evaluation value is greater than or equal to a threshold value, and input data based on the linearity evaluation value when the linearity evaluation value is determined to be greater than or equal to the threshold value in the linearity evaluation value determination unit A smoothed line determination unit that determines a flattened or smoothed line, and a minor axis direction parameter automatic that corrects the value of the parameter when the linearity evaluation value determination unit determines that the linearity evaluation value is smaller than a threshold value. It is composed of a change unit.

[0014]

When the data represented by a plurality of points distributed on the coordinate plane is input, the linearity evaluation value is calculated, and the angle of the major axis of the ellipse that maximizes this value is determined. Data smoothing is performed to generate a smoothed line of the input data. At that time, the parameter corresponding to the length of the ellipse's minor axis of the ellipse membership function is automatically adjusted according to the type and magnitude of noise, so that smoothing is performed appropriately even for data containing unknown noise. The generated line is generated.

The automatic adjustment of the above parameters can be performed, for example, by
When the linearity evaluation value is smaller than a certain threshold value, the parameter is multiplied by an appropriate coefficient.

[0016]

1 shows the configuration of a data processing system including a data smoothing device of the present invention. This data processing system is a data smoothing device 1 according to the present invention.
And a signal for displaying the processing result of the data smoothing device 1 and a data input device 2 for inputting data represented by a dot sequence described later to this device and recording the result on a predetermined sheet such as a CRT. Result output device 3 for outputting
It consists of and.

In the data smoothing device 1, the linearity evaluation value calculation unit 11 for executing the processing operation described later to calculate the linearity evaluation value E (θ), and the linearity evaluation value E obtained thereby are appropriate. A linearity evaluation value determination unit 12 that determines whether or not (specifically, whether or not the linearity evaluation value is equal to or greater than a threshold value), and a data point based on the linearity evaluation value when the determination result is equal to or greater than the threshold value. When the determination result of the smoothed line determination unit 13 that determines the line in which the row is flattened or smoothed and the linearity evaluation value determination unit 12 is smaller than the threshold value, fuzzy in the minor axis direction of the three-dimensional elliptic membership function described later. It is provided with a short axis direction parameter automatic changing unit 14 that automatically corrects the parameter a _y that determines the entropy. Such a data smoothing device 1 is composed of a CPU in which processing procedures for realizing the functions of the above-described respective constituent elements are stored in advance.

The data processing system shown in FIG. 1 is used, for example, as a means for obtaining a smooth graph in which noise and variations are removed from plant observation data in plant control and displaying it on a display. In this case, the data input device 2 is composed of a measuring instrument that generates various measured values in the plant as time series data.

The data smoothing device 1 has the above-described configuration and performs the data processing operation shown in FIG.

First, when data represented by a point sequence as described later is input from the data input device 2 (step 1), the linearity evaluation value calculation unit 11 determines the linearity evaluation value E (step 2). ). This determination method will be described later in detail.

Next, the linearity evaluation value determination unit 12 determines whether or not the linearity evaluation value E is above a certain threshold value α _s (step 3). This threshold value α _s determines the fineness of smoothing, and is set according to the type and level of noise as described above.

According to the above judgment, when the linearity evaluation value E is equal to or _larger than the threshold value α _s , the linearity evaluation value E is sent to the smoothed line determining section 13, where the linearity evaluation value E is maximized. The angle ψ is obtained to determine a line obtained by flattening or smoothing the input data point sequence (step 4), outputting a signal representing it (step 5), and returning to the state of waiting for data input.

On the other hand, when the linearity evaluation value E is smaller than the threshold value α _s , the linearity evaluation value E obtained by the linearity evaluation value calculation unit 11 is sent to the short-axis direction parameter automatic changing unit 14 to be described later. The parameter a _y which determines the fuzzy entropy in the minor axis direction of the dimensional elliptic membership function is automatically modified (step 6). Here, the parameter a _y is multiplied by a coefficient k (an appropriate value of about 1.1 to 2.0) to correct its size. After this correction, return to step 2.

Next, in order to describe the data processing in the data smoothing device 1 in more detail, an angle determination method based on the linearity evaluation value will be described. As described in Japanese Patent Application No. 2-243353 filed earlier, this is to measure the angle of a line connecting point sequences by using a three-dimensional elliptic membership function that defines a fuzzy set. Is as follows.

In general, a linear relationship (correlation) between two variables is often displayed in the form of a distribution chart in which the two variables are plotted on the horizontal axis and the vertical axis and individual data are plotted on the coordinates.

On the other hand, when the bivariate normal distribution data has an elliptic contour, the human judgment on the correlation between the two variables is based on the elliptic contour (ratio of the minor axis to the major axis of the elliptical contour). It is known to

On the basis of this, in the embodiment, in order to evaluate the linearity of the straight line of the linear point sequence, that is, the linearly plotted point sequence, an elliptical "measurement" is set in the coordinate space, An algorithm that superimposes the long axis of the ellipse and the data point sequence of the measurement target is used.

First, the xy Cartesian coordinates and the fuzzy operation are suitable.
A three-dimensional space consisting of degrees is defined, and on the xy plane
Point sequence P plotted linearly₁(x₁ , Y₁), P₂(x
₂ , Y ₂) 、 ・ ・ ・ ・ ・ 、 P_n(x_n , Y_n).

The fact that the data having a bivariate normal distribution has an elliptical contour means that the product of two bell-shaped membership functions (represented by a function isomorphic to the normal distribution) forms a three-dimensional elliptic membership function. It has the same mathematical meaning as doing. Based on this, a three-dimensional elliptic membership function represented by the following equation (1) is set for linearity evaluation of the point sequence. In addition, here, the long axis of the ellipse is placed on the x-axis,
Place the minor axis of the ellipse on the y-axis.

[0030] t = exp [- {(x / a x) 2 + (y / a y) 2}] ... (1) Next, the center of the ellipse, critical assessment of the most linear in the point sequence Move to a point to see. Usually, the point near the middle of the n points to be measured is made to coincide. The coordinates of the center of the ellipse at this time are (A, B), and the ellipse is rotated around this point. When the rotation angle at this time is θ, the three-dimensional elliptic membership function is as follows.

[0031] t = M (x, y, θ) = exp [- {(X / a x) 2 + (Y / a y) 2}] ... (2) where, X = (x-A) cosθ + ( y−B) sin θ (3) Y = − (x−A) sin θ + (y−B) cos θ (4) The above equation is shown in FIG. In this case, the center of the ellipse is the point P ₄ , and (A) and (B) indicate the times when the ellipse is rotated by the angles θ and θ ′, respectively. The linearity evaluation of the point sequence P _i (i = 1, 2, ...) In this case is referred to as “type 1”.

Next, when the parameter of the x-axis is a parameter that does not affect the dispersion of the point sequence (for example, when the x-axis is the time axis and the time is accurately measured), it will be described later. The linearity evaluation value of should depend only on the value of y. That is, only the major axis of the ellipse should rotate, the minor axis should remain vertical, and
The shape of the ellipse membership function projected on the x-axis should not change due to the rotation of the ellipse.
In this case, the three-dimensional elliptic membership function is as follows.

T = M (x, y, θ) = exp [− {(X / a _x ') ² + (Y / a _y ) ² }] (5) where X = (x−A) / cosθ ... (6) Y = - (x-a) tanθ + (y-B) ... (7) a x '= a x / cosθ ... (8) expressed in FIG equation in this case, as shown in FIG. 4 Become. In the figure, (A) and (B) represent the ellipses at angles θ and θ ', respectively.
Indicates when rotated. The linearity evaluation in this case is referred to as "type 2".

As described above, rotating the three-dimensional elliptic membership function corresponds to an operation of superimposing an elliptic "measurement" on a sequence of points, which allows ambiguity of ambiguous lines.

Therefore, the sum T of the goodnesses of fit t ₁ , t ₂ , ..., T _n by the three-dimensional elliptic membership function of each measurement object point when the ellipse "measurement" is adjusted to an arbitrary angle θ.
(Θ) and as shown in FIGS. 5 (A) and 5 (B), the fitness t _S1 , t _S2 , ..., By the three-dimensional elliptic membership function of the points obtained by projecting the points to be measured on the major axis of the ellipse. sum of t _Sn S (θ)
Is defined as the linearity evaluation value E (θ), and the angle ψ of the major axis that maximizes this value is adopted as the angle of the ambiguous line. That is, the following calculation is performed.

[0036]

[Equation 1]

At this time, if a sufficiently satisfactory evaluation value is not obtained, it is effective to move the position of the center of the ellipse. Usually, it is an efficient method to roughly evaluate linearity in advance for several points near the center of the point to be measured and to evaluate in detail the central point where the highest linearity is obtained.

This method can be adapted to the human sense by changing the lengths of the major axis and the minor axis of the ellipse.

Further, in the method using the elliptical "measurement", the linearity evaluation value E (ψ) at the angle ψ obtained by the above equation (10) is a certainty factor of the linearity of the ambiguous line to be measured. At the same time, by being normalized to a value in the range of 0 to 1, it is also given as the confidence factor CF for the angle. E
As is clear from the definition of (ψ), the certainty factor CF has a maximum value of 1 when the point sequences are perfectly aligned.

In the data smoothing based on the angle measurement as described above, basically, the angle measurement by the three-dimensional elliptic membership function is repeatedly performed in order from the end with respect to the point sequence to be smoothed.

In the processing operation of FIG. 2, when the parameter a _y that is corrected when the linearity evaluation value E determined as described above is smaller than the threshold value α _s is large, large noise such as spike noise is also picked up. Parameter a
It is desirable to make _y as small as possible.

Finally, an example of data smoothing according to the present invention will be shown.

FIG. 6 shows that when the threshold value α _{s is} 0.70,
FIG. 7 shows the smoothing result when the threshold value α _s = 0.85. In these figures, the data to be smoothed is shown by the alternate long and short dash line, and the line obtained by the smoothing process of the present invention is shown by the solid line.

[0044]

As described above, according to the present invention, in data smoothing that achieves noise reduction of a signal from a system that cannot be described by a mathematical model, the type and level of noise contained in data to be smoothed. Since it is not necessary to check in advance or to tune the parameters that determine the parameters in the minor axis direction of the ellipse, smoothing of data containing unknown noise can be performed automatically and accurately.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a data processing system including a data processing device of the present invention.

FIG. 2 is a flowchart showing a data processing operation of the embodiment.

FIG. 3 is a diagram showing a state in which an ellipse of a three-dimensional ellipse membership function used in the embodiment is rotated.

FIG. 4 is a diagram showing a case where an ellipse of an ellipse membership function is rotated only with its major axis rotated and its minor axis kept vertical.

FIG. 5 is an explanatory diagram when a plurality of points are projected on the major axis of an ellipse.

FIG. 6 is a diagram showing an example of data smoothing according to the present invention.

7 is a diagram showing an example of data smoothing in which the threshold value of the linearity evaluation value is different from that in FIG.

FIG. 8 is a diagram showing an example of time series data disturbed by non-stationary noise and exceptional noise.

9 is a diagram showing an example of smoothing in the case where the parameter in the minor axis direction of the ellipse is inappropriate for the time series data of FIG.

[Explanation of symbols]

1 ... Data smoothing device, 2 ... Data input device, 3
... processing result output device, 11 ... linearity evaluation value calculation unit, 12
... Linearity evaluation value determination unit, 13 ... Smoothed line determination unit, 14 ... Parameter automatic change unit.

Claims (2)

[Claims] 1. When data represented by a plurality of points distributed on a coordinate plane is input, the center of an ellipse of a predetermined elliptic membership function is near the center of a point sequence selected from the plurality of points. , The ellipse is rotated about that point, and the sum of goodness of fit by the ellipse membership function of each point at any rotation angle and the point projected on the major axis of the ellipse. A ratio with the sum of the goodness of fit by the ellipse membership function is calculated, the ratio is used as the linearity evaluation value of the data, and the angle of the major axis of the ellipse at which the linearity evaluation value becomes maximum is measured, and the first angle measurement Sometimes, a line that passes through the center of the ellipse at that time and has the angle is drawn up to near the middle of the center of the ellipse and the center of the ellipse in the next angle measurement, and then the line obtained by the previous angle measurement is drawn. Angle from the end point to that time In a data smoothing device that outputs a line obtained by smoothing the data by sequentially repeating the operation of drawing a straight line to the end point of the line obtained by the constant, a parameter corresponding to the length of the minor axis of the ellipse of the ellipse membership function. Is automatically adjusted according to the type and magnitude of noise to generate a line that is appropriately smoothed for data containing unknown noise. 2. A linearity evaluation value calculation unit for calculating the linearity evaluation value for the point sequence of the data, and whether the linearity evaluation value obtained by the linearity evaluation value calculation unit is greater than or equal to a threshold value. And a linearity evaluation value determining unit that determines whether the data point sequence is flattened or smoothed based on the linearity evaluation value when the linearity evaluation value is determined to be a threshold value or more in the linearity evaluation value determination unit. A smoothed line determination unit that determines a line, and a linear parameter evaluation unit that corrects the value of the parameter when the linearity evaluation value determination unit determines that the linearity evaluation value is smaller than a threshold value. The data smoothing device according to claim 1, wherein