JP6182473B2 - Data analysis apparatus, method and program - Google Patents

Description

  The present invention relates to a data analysis apparatus, method, and program, and in particular, a technique for approximating two-dimensional data in computer graphics with a polygonal line, an upwardly convex or downwardly convex curve, and an outlier in data analysis, The present invention relates to a data analysis apparatus, method, and program for early detection of change points, curved areas, and straight line areas.

  As a first conventional technique, a change point detection method based on the statistical test of Non-Patent Document 1 will be described.

  Data series now

があり、このデータ系列中に変化点候補を１つ設ける。この点をｔとすると変化点の前後で２つのデータ系列に分割する。前半を、

There is one change point candidate in this data series. If this point is t, it is divided into two data series before and after the change point. The first half

後半を

The second half

とする。ここで、各x_j(j=1,2,…,n)は多次元連続値を取るものとする。ここで各時刻ｔにおいて

And Here, each x _j (j = 1, 2,..., N) is assumed to take a multidimensional continuous value. Here at each time t

が与えられたときのx_tの条件付確率密度関数

Conditional probability density function of x _t given

を、

The

とする。ここでΣは実数値パラメータを要素とする分散共分散行列を表す。上付きＴは転置を表す。w_tとしては自己回帰モデルとして、

And Here, Σ represents a variance-covariance matrix whose elements are real-valued parameters. The superscript T represents transposition. w _t is an autoregressive model,

とする。あるいは１次元の場合には多項式回帰モデルとして、

And Or in the case of one dimension, as a polynomial regression model,

とする。

And

Parameters α ₁ ,..., Α _k , μ are _obtained by maximum likelihood estimation. The result of substituting the parameters obtained by maximum likelihood estimation into Equation (2) and Equation (3)

とする。ここで、データ

And Where the data

とモデルによる予測値との２乗誤差

Error between model and model predicted value

は、

Is

と表される。

It is expressed.

  In order to obtain the change point, for a predetermined threshold δ,

が成り立つt*が求める変化点となる。ここでt*は

The t * that holds is the change point to be obtained. Where t * is

である。

It is.

  To detect multiple change points,

と

When

をそれぞれ前半と後半に分割し再帰的に変化点を検出すればよい。

May be divided into the first half and the second half, and the change point may be detected recursively.

  As a second prior art, a method for approximating a segment, which is a pixel row, in a computer graphics line segment will be described.

  As a method of approximating a segment to a line segment, there is a binary decomposition method (for example, see Non-Patent Document 2). As a procedure, the following procedure is performed recursively.

  1. The distance is calculated from each pixel with respect to a straight line connecting the start point and end point of the image sequence.

  2. If the maximum distance is below a certain threshold, the straight line is adopted as the line segment. Otherwise, the pixel column is divided by the pixel, and the above 1.. This process is repeated recursively.

  3. If there is no division of the pixel column, the process is terminated.

  If the original pixel row is a closed curve, the starting point is arbitrarily selected, the point farthest from that point is selected as the end point, and the above procedure is performed.

  As a third conventional technique, a method for detecting a corner point of a pixel row will be described.

  It is preferable that the dividing points of the pixel column are corner points because the features of the figure are well understood. The corner detection method (corner detection) described below (see, for example, Non-Patent Document 3) is a method of positively finding a corner point of a pixel row and using that point as a partitioning break.

1. The starting point P ₁ to the end point P _{n of the} pixel row are assumed.

  2. i = k.

3. Determining the angle of a line connecting the pixels P _ik and P _{i + k} points before and after a distance k pixels for a point Pi.

  4). If i <n−k, i = k + 1.

5. The P _i was at an angle of less minima than the threshold value and the division point.

  Although there is no particular rule on how to select k, it is preferable to first select a large value for k and gradually decrease it in order to capture the features of the figure.

V. Guralnik and J. Srivastava, Event detection from time series data, In Proceedings of the Fifth ACM SIGKDD, International conference on Knowledge Discovery and Data Mining (KDD99), ACM Press, pp. 32-42, 1999. Digital Image Processing Editorial Board, Digital Image Processing, CG-ARTS Association), pp. 189-190 Digital Image Processing Editorial Committee, Digital Image Processing, CG-ARTS Association), page 190 Takeshi Nishina, Statistical Process Control, Asakura Shoten, pages 24 to 56, November 25, 2009

  However, the first prior art described above detects only a change point, and there is a problem that it cannot be determined whether a region other than the change point is linear or curved. In general, analysis is performed by excluding outliers that are greatly deviated from other values, but this technique has a problem that outliers cannot be excluded. In addition, the technique has a large amount of calculation in which parameter estimation is performed for all candidates to calculate the above-mentioned t * and a square error must be calculated, and a guideline is provided for determining the threshold value δ. Therefore, there is a problem that it is not possible to clarify what guideline the obtained change point is calculated based on. Another problem is that when new data is available, it must be recalculated including the existing data.

  Also, the second conventional technique implicitly assumes that the pixel column has a small deviation from the linear approximation. If the deviation from the linear approximation is large (when the data is fluctuating finely), there is a problem that the procedure does not end without converging. Although it is necessary to set the threshold value of the maximum distance well, there is a problem that this must be implemented by trial and error.

  The third prior art also has a problem that it is difficult to properly select k at a point separated by k pixels, a problem that it is difficult to set an angle threshold, and a difficulty in setting an area when it is determined to be a minimum. There are challenges.

  In all the above prior arts, it is impossible to extract outliers and curve areas. In addition, there is a problem that it is not possible to detect a straight line region or a change point in consideration of fluctuations of data.

  The present invention has been made in view of the above points. When data is given in two dimensions, an outlier, a change point at which the data changes, a curved area, and a linear area are detected, and the detection is performed. Provided is a data analysis apparatus, method, and program that can detect the points and areas at an early stage not only for existing data but also when data is obtained sequentially, with a small amount of calculation. For the purpose.

A data analysis device for analyzing a plurality of two-dimensional data having an order relationship ,
Create a vector from two-dimensional data adjacent in the plurality of sequential relationship of the two-dimensional data, the angle of adjacent contact vector, and the vector angle generating means for calculation using vectors cross product,
A statistical value calculating means for obtaining an average of the angle formed and a standard deviation of the angle formed;
A confidence interval is calculated from the average of the angles formed and the standard deviation of the angles formed, and depending on whether or not each formed angle is included in the confidence interval, the two-dimensional data that is the calculation source of the angle formed is an outlier or Confidence interval determination means for determining whether or not a change point candidate;
Is provided.

  According to one aspect, two adjacent points are detected in order to detect a change point, a linear region, a curved region, and an outlier using an angle formed by two vectors and an average thereof, or a sum of angles formed by two vectors. A vector is created from two pieces of data, and it is possible to determine whether a point other than the change point is a straight line region or a curved region by changing the sum of angles formed by the vectors, and it is also possible to eliminate outliers.

It is a structural example of the outlier and change point detection apparatus in the 1st Embodiment of this invention. It is a flowchart of the process of the outlier and change point detection apparatus in the 1st Embodiment of this invention. It is an image figure of the angle which the vector created from the data in the 1st Embodiment of this invention and a vector makes. It is a structural example of the data analyzer in the 2nd Embodiment of this invention. It is an example of the closed curve in the 2nd Embodiment of this invention. It is an example of the division | segmentation of the closed curve in the 2nd Embodiment of this invention. It is an example of the division | segmentation of the closed curve which has the continuous end point in the 2nd Embodiment of this invention. It is a flowchart of the pre-processing in the 2nd Embodiment of this invention. It is a figure explaining the matching of the number in the 2nd Embodiment of this invention. It is a flowchart of the curve area | region detection in the 2nd Embodiment of this invention. It is a flowchart of the change point detection in the 2nd Embodiment of this invention. It is a flowchart of the outlier detection in the 2nd Embodiment of this invention. It is a flowchart of the linear area | region detection in the 2nd Embodiment of this invention. It is a figure explaining the matching of the number in the 2nd Embodiment of this invention geometrically.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
The data changes suddenly before and after the change point, and even if there is a change at a point other than the change point, the change is small. If this small fluctuation is ignored, it can be said that the data transitions linearly with respect to time. The case where it can be converted so as to make a linear transition by performing some conversion even when it is not so is handled. In other words, since the data can be approximated by a line in a piecewise manner, it can be said that the data always grows in the same direction with respect to the passage of time except for the fluctuation portion.

The present invention solves the problem based on this idea. That is, a vector r _i = (t _{i + 1} −t _i , x _{i + 1} −x _i ) is created from two adjacent data (t _i , x _i ) and (t _{i + 1} , x _{i + 1} ). and, calculating a vector cross product with the neighboring vector between (r _i and r _{i + 1).} An angle formed by two vectors is calculated from the vector outer product, and a confidence interval of the angle formed is obtained. If the confidence interval is outside the confidence interval, an outlier or a change point candidate is determined. In order to be a conversion point, an average value is obtained as a moving average from data of N corners, and when it is outside the confidence interval, it is detected as a change point rather than an outlier.

  FIG. 1 is a configuration example of an outlier and change point detection apparatus according to the first embodiment of the present invention.

  The outlier and change point detection apparatus 100 shown in FIG. 1 includes a data input unit 110, a preprocessing unit 120, a vector generation unit 130, a formed corner generation unit 140, a data storage unit 150, a statistical value calculation unit 160, and a parameter input unit 170. A confidence interval determination unit 180 and a result output unit 190.

The data input unit 110 inputs time-series data (t _i , x _i ), i = 0, 1, 2,..., N (t _i is time data, x _i is an observed value at time t _i ), and data Store in the storage unit 150. Examples of the data input method include, but are not particularly limited to, input from a terminal and reading stored data such as a database.

The pre-processing unit 120 smoothes the input data (t _i , x _i ) stored in the data storage unit 150 by moving average in order to sufficiently reduce the variance of the angle θ _i formed. The pre-processing unit 120 operates when the standard deviation of the angle formed by the vector is large in the statistical value calculation unit 160.

  The vector creation unit 130 reads data from the data storage unit 150 and creates a vector from two adjacent data.

The formed angle creation unit 140 obtains an angle θ _i formed by the two vectors by obtaining an outer product of the vectors of the vector data created by the vector creation unit 130 and stores them in the data storage unit 150.

The statistical value calculation unit 160 acquires the angle θ _i formed by the vector from the data storage unit 150, and obtains the average value μ and the standard deviation σ of the formed angle θ _i .

  The parameter input unit 170 acquires a parameter for determining a confidence interval. The parameters are input from the operator's terminal.

The confidence interval determination unit 180 acquires the angle θ _i formed by the two vectors from the data storage unit 150, determines whether the angle θ _i formed by the input parameter is included in the confidence interval using the input parameters, and includes If not, it is determined as an outlier, the statistical value calculation unit 160 is requested to calculate an average value, a confidence interval for the average value is obtained, and it is determined whether the average value is included in the average confidence interval. If it is not included, it is determined as a change point.

  FIG. 2 is a flowchart of processing of the outlier and change point detection apparatus according to the first embodiment of the present invention.

  First, a description will be given of how to create a vector from data input and obtain an angle formed by two vectors.

  Parameters for determining outliers are input. (Step 101).

Now, it is assumed that time series data (t _i , x _i ) (i = 0, 1, 2,..., N) is obtained in the data input unit 110 and new data is added every moment. The obtained data is stored in the data storage unit 150 (step 102). Here, t _i is time data, and x _i is a value observed at time t _i .

In step 103, it is determined whether pre-processing is necessary or not. If pre-processing is necessary, the process proceeds to step 104, and if not necessary, the process proceeds to step 105. If the standard deviation is greater than or equal to the predetermined value in step 108, the pre-processing is performed in step 109 because the pre-processing is necessary (step 104). In the pre-processing, the pre-processing unit 120 smoothes the original data (t _i , x _i ), i = 1, 2,.

As a pre-processing method in the pre-processing unit 120, for example, as a smoothing method, a method of taking a moving average with respect to original data (t _i , x _i ) (i = 0, 1, 2,..., N) Alternatively, as a method for sufficiently reducing the formed angle θ _i , a method of multiplying the time t by 100, for example, and a value x of 1/100, for example, can be considered. If the early data is not in a steady state (mean and standard deviation are close to 0), all the angles θ _i (i = 1, 2, ..., n-1) formed by the vectors There is a method to use θ _i (i = m, m + 1, ..., n-1) and not use θ _i , i = 1,2, ..., m-1 instead of using .

The vector creation unit 130 reads data from the data storage unit 150 and creates the following vector r _i (i = 1, 2,..., N) from two adjacent data (step 105).

得られたベクトルデータをなす角作成部１４０に渡し、なす角作成部１４０にてベクトル外積r_i×r_i+1,(i=1,2,…,n-1)を計算し、その２つのベクトルのなす角θ_i（i=1,2,…,n-1）を求める（ステップ１０６）。なす角を求めるには、次のようにして行う。ベクトル外積の定義から

The obtained vector data is transferred to the corner creation unit 140, which calculates the vector outer product r _i × r _{i + 1} , (i = 1, 2,..., N-1), An angle θ _i (i = 1, 2,..., N−1) formed by two vectors is obtained (step 106). The angle to be formed is determined as follows. From the definition of vector outer product

が成り立つ。結果、

Holds. result,

が得られる。よって２つのベクトルのなす角は、

Is obtained. So the angle between the two vectors is

と求められる。但し、

Is required. However,

である。得られたなす角のデータをデータ蓄積部１５０に格納する。

It is. The obtained angle data is stored in the data storage unit 150.

  FIG. 3 shows an image of a method of creating a vector from adjacent data by the vector creation unit 130 and creating an angle formed by the adjacent vector in the formed corner creation unit 140. Since the vector outer product is a vector, the direction of the angle formed is also shown in FIG.

Next, the statistical value calculation unit 160 reads the angle data formed from the data storage unit 150 and obtains the average and standard deviation of the angle θ _i formed by the two vectors (step 107). Now, _assuming that the latest data is (t _n , x _n ), θ ₁ , θ ₂ ,..., Θ _n-1 are obtained by the method described above, and the average μ (1, n-1) is obtained. Ask.

標準偏差σ(1,n-1)は、

The standard deviation σ (1, n-1) is

として求める。

Asking.

The angle θ _i formed by the two vectors has a normal distribution N (μ (1, n-1), σ (1,1) with an average μ (1, n-1) and standard deviation σ (1, n-1). n-1)). This is the average value of the angle θ _i

標準偏差

standard deviation

であることを仮定している。元データがほぼ直線的であれば平均値

Is assumed. Average value if source data is almost linear

は成り立つ。標準偏差が所定の値以上である場合には（ステップ１０８,Yes）、ステップ１０３に移行し、ステップ１０４において前述の事前処理を行う。

Holds. If the standard deviation is equal to or larger than the predetermined value (step 108, Yes), the process proceeds to step 103, and the above-described pre-processing is performed in step 104.

In the following description, each θ _i formed by two vectors has a normal distribution N (μ (1, n-1) having an average μ (1, n-1) and a standard deviation σ (1, n-1). , Σ (1, n−1)), and the early data is assumed to be in a steady state.

  The change point can be regarded as a point where an outlier starts to appear at a high frequency. Outliers are candidates for change points. Therefore, first, a method for detecting an outlier will be described.

Θ _i (i = 1, 2,..., N−1) is obtained from the data storage unit 150. Since it is assumed that the angle θ _i formed by the two vectors follows a normal distribution, in the confidence interval determination unit 180, θ _m (where m <n−1) is a confidence interval.

に含まれないときのθ_mを外れ値とする。ここでZはパラメータ入力部１７０から入力されるパラメータであり、95%信頼区間の場合、正規分布表からZ=1.96、99%信頼区間の場合Z=2,58となる。信頼区間の計算では、θ_mのデータは使用せず、その１つ前のデータまでを用いて計算してある。外れ値でなければステップ１０３に戻る（ステップ１１０、No）。

Let θ _m when it is not included in be an outlier. Here, Z is a parameter input from the parameter input unit 170. In the case of 95% confidence interval, Z = 1.96 from the normal distribution table, and in the case of 99% confidence interval, Z = 2,58. In the calculation of the confidence interval, the data of θ _m is not used, and the calculation is performed up to the previous data. If it is not an outlier, the process returns to Step 103 (Step 110, No).

  If it is an outlier (step 110, Yes), the statistical value calculation unit 160 calculates the average value (step 111), the confidence interval determination unit 180 obtains a confidence interval for the average value, and the average value is the confidence interval. It is determined whether it is in. This process will be described in detail below.

It is determined whether θ _m is an outlier that is not a change point or a change point (steps 110 and 112). When it is not a change point but an outlier (step 110, Yes, step 112, No), an outlier is output. As N ≧ 3,

である。なぜならば、θ_mを外れ値であるとは、元データ（t _m+1,x _m+1）が外れ値であることが原因であり、式(11)から元データ（t_m+1,x _m+1）が外れ値であるとθ_m，θ_m+1，θ_m+2の値に影響を与えることがわかる。しかし、θ_mが外れ値であれば

It is. This is because θ _m is an outlier because the original data (t _{m + 1} , x _{m + 1} ) is an outlier, and the original data (t _{m + 1} , It can be seen that if x _{m + 1} ) is an outlier, the values of θ _m , θ _{m + 1} , and θ _{m + 2} are affected. However, if θ _m is an outlier,

である。一方、θ_mが変化点であれば、m以降のθ_i，i>mについても式(15)を満たさないことが多いため、|μ(m,m+N-1)|>0となり、θ_mが変化点であるときの平均の絶対値は、θ_mが外れ値であるときの平均の絶対値よりも大きいことが考えられる。ここで次のような定理が知られている。

It is. On the other hand, if θ _m is the changing point, θ _i and i> m after m often do not satisfy Equation (15), so | μ (m, m + N−1) |> 0, It is conceivable that the average absolute value when θ _m is a change point is larger than the average absolute value when θ _m is an outlier. Here, the following theorems are known.

  Theorem: if X follows a normal distribution with mean μ and standard deviation σ, mean based on a random sample of size N

は、平均μ、標準偏差

Is the mean μ, standard deviation

の正規分布に従う。

Follow the normal distribution of.

If θ _m is the changing point, the average μ (m, m + N-1) of the N points after m for N ≧ 3 does not satisfy Equation (17), and θ _m is an outlier that is not a changing point. Equation (17) is satisfied. Thus, it is determined whether it is a change point or an outlier. Parameter Z in equation (17) is the same as that in equation (15). This Z is input from the parameter input unit 170.

より早く変化点を検出したい場合には、式(15)において信頼区間を連続するデータでプラス側あるいはマイナス側で同じ側にはみ出したときには変化点とみなしてもよい。

When it is desired to detect the change point earlier, it may be regarded as a change point when the confidence interval in Formula (15) protrudes to the same side on the plus side or the minus side with continuous data.

  When it is not a change point but an outlier (step 110, Yes, step 112, No), the result output unit 190 outputs outlier data (step 113).

  If it is an outlier (step 110, Yes) and it is a change point (step 112, Yes), the result output unit 190 excludes data up to the change point or data next to the change point from the analysis target. In other words, the data used for calculating the average and variance in the equation (15) is updated from the data next to the change point or the data ahead of the change point, and the data in the data storage unit 150 is updated all at once (step 114). ).

  The result output unit 190 outputs change point data (step 115). If new data is input, the process proceeds to step 102 and the above processing is repeated.

  According to the present embodiment, in order to detect outliers and change point candidates of data, adjacent data is treated as a vector, an outer product of the vectors is obtained, and an average of the formed angles and a standard deviation of the formed angles are obtained. It is only a calculation and the amount of calculation is small. The necessary parameters are easy because the number of data required for averaging and what percentage of confidence intervals are set.

[Second Embodiment]
In the present embodiment, change points, straight line regions, curved regions, and outliers are detected for two-dimensional data in computer graphics.

  FIG. 4 shows a configuration example of the data analysis apparatus according to the second embodiment of the present invention.

  The data analysis apparatus 200 shown in the figure includes a data input unit 210, a management chart data creation unit 220, a data storage unit 230, a calculation unit 240, a closed curve division unit 250, a result output unit 260, and a control unit 270. The creation unit 220 includes a vector creation unit 221, a sum of corners creation unit 222, and a group creation unit 223. The calculation unit 240 calculates the group average value and the range R (range: difference between the maximum value and the minimum value). An R control chart calculation unit 241 that calculates an R control chart to be managed, and manages measured values in time series

管理図（以下では、X⁻管理図と記す）を計算するX⁻管理図計算部２４２を有する。

An X ^- control chart calculation unit 242 for calculating a control chart (hereinafter referred to as X ^- control chart) is provided.

  The data input unit 210 acquires two-dimensional data and stores it in the data storage unit 230 via the control unit 270. The data acquisition method includes input from a terminal, reading stored data such as a database, and the like, but the input method is not particularly limited.

  The data storage unit 230 stores the data input from the data input unit 210 and the results obtained by the control chart data creation unit 220, the calculation unit 240, and the closed curve division unit 250.

  The result output unit 260 outputs the results obtained by the control chart data creation unit 220, the calculation unit 240, and the closed curve division unit 250.

[1] Computer graphics specific processing:
In the case of application to computer graphics, there are cases where a figure to be drawn has a plurality of d _k for the same t _k as a closed curve. In this case, the closed curve dividing unit 250 performs the following processing. Since with one d _k for the same t _k in the data analysis, the contents to be described closed curve division section 250 and the following is not required.

Closed curve, i.e. the data (t _k, d _k) if it has a plurality of d _k for the same t _k in the need to separate the data to have a d _k for the same t _k to a plurality of sets There is. The method of dividing will be described.

FIG. 5 shows an example of dividing a closed curve in the second embodiment of the present invention. In FIG. 5, the vertical axis represents d _k and the horizontal axis represents t _k . In the case of a closed curve when d _k is not continuous with respect to the same t _k as shown in FIG. 5, the number of d _k is counted for each t _k (however, when it is continuous, it is counted as one). In FIG. 5, the number written on the line parallel to the d _k axis is the number of d _k on the line, and the number written between the parallel lines is the number of d _{k in} that region. Is a number.

Now, we sort by the numbers written between these parallel lines. As a result, a region in which three “6” s in the center are continuous is one region. Numbering is performed for each group as shown in FIG. 6 in the region having the largest number of d _k (“6” in the case of FIG. 5). In FIG. 6, L1, L2,..., L6. These sets are expanded to areas _where the number of d _k is “4” or “2”. The extension method takes a region (d _k −ε, d _k + ε) for each set of d _k values, and the sets included in the region are recognized as the same set. FIG. 6 shows an example of a region _where the number of d _k is “4”, which corresponds to L1, L3, L5, and L6. In this way, the set is divided so that d _k becomes one for t _k .

Next, consider the case of a closed curve when d _k continues for the same t _k as shown in FIG.

As shown in FIG. 7, the end points other than the side where d _k is continuous are deleted and divided into sets. The grouping method is the same as in the case of the closed curve when d _k is not continuous for the same t _k . The deleted portion is connected again with a straight line after the processing described below is completed.

  This is the processing unique to computer graphics.

  Hereinafter, processing in data analysis and processing in computer graphics after finishing the above processing will be described.

[2] Data analysis processing
[2-1] Pre-processing FIG. 8 is a flowchart of the pre-processing in the second embodiment of the present invention.

  First, pre-processing from data input to creation of a vector, creation of a sum of angles formed by two vectors, and creation of a group in a Schuhart control chart (see, for example, Non-Patent Document 4) will be described.

  The pre-processing is mainly performed by the control chart data creation unit 220 shown in FIG.

Now, time-series data (t _i , d _i ), i = 1, 2,..., N are obtained from the data input unit 210 via the control unit 270, and new data is constantly added to the data storage unit 230. Assume that the user is participating (step 201). Here, t _i is time data and d _i is a value observed at time t _i .

The vector creation unit 221 reads data (t _i , d _i ) from the data storage unit 230 and creates the following vectors v _i (i = 1, 2, 3,..., N) from two adjacent data. (Step 202).

ベクトル作成部２２１は、得られたベクトルをなす角の和作成部２２２に送り、なす角の和作成部２２２は、ベクトルの外積

The vector creation unit 221 sends the obtained vector to the corner sum creation unit 222. The corner creation unit 222 creates the outer product of the vectors.

を計算し（ステップ２０３）、その２つのベクトルのなす角θ_i (i=3,4,…,n)を求める（ステップ２０４）。なす角を求めるには、次のようにして行う。ベクトルの外積の定義から、

Is calculated (step 203), and an angle θ _i (i = 3,4,..., N) formed by the two vectors is obtained (step 204). The angle to be formed is determined as follows. From the definition of vector cross product,

が成り立つ。結果、

Holds. result,

が得られる。隣接するデータからベクトルを作成し、隣接するベクトルのなす角を作成するイメージ図を図３に示す。ベクトル外積はベクトルであるためなす角についても方向があることも図３に示している。

Is obtained. FIG. 3 shows an image diagram in which a vector is created from adjacent data and an angle formed by the adjacent vector is created. Since the vector outer product is a vector, it is also shown in FIG.

  So the angle between the two vectors is

と求められる。但し、

Is required. However,

である。なす角の和

It is. Sum of corners

は、

Is

として求める（ステップ１０５）。

(Step 105).

The group creation unit 223 creates a group used in the X ^-- R control chart (step 206). The creation method will be described. The group size (the number of data in one group) will be described as “2”.

If the group is g ₄ , g ₅ ,…, g _n ,

となる。以後、一つ一つの群を「点」と表現する。得られた群の点は、データ蓄積部２３０に記憶される。このg₄,g₅,…,g_nは、分析前においては曲線領域、変化点、外れ点、直線領域いずれにも該当していないので、未分類点とする（ステップ２０７）。

It becomes. Hereinafter, each group is expressed as a “point”. The obtained group points are stored in the data storage unit 230. These g ₄ , g ₅ ,..., G _n do not correspond to any of the curved region, change point, outlier point, and straight line region before the analysis, and are therefore unclassified points (step 207).

Where original data d _k , vector r _k , sum of angles formed

群g_kの番号付けの対応の例を図９に示す。元データd_k,d_k+1からベクトルr_kが作成される。ベクトルr_k、r_k+1からなす角の和

FIG. 9 shows an example of the correspondence of the numbering of the group g _k . A vector r _k is created from the original data d _k and d _{k + 1} . Sum of angles from vectors r _k , r _{k + 1}

が作成される。

Is created.

から群g_kが作成され、データ蓄積部２３０に格納する。

The group g _k is created from and stored in the data storage unit 230.

[3] Detection of Curve Area, Change Point, Outlier, and Linear Area Next, a method for detecting the curve area, change point, outlier, and straight line area by the calculation unit 240 will be described.

  As a detection order, first, a curved region is detected, then a change point is detected, an outlier is detected, and finally a straight region is detected.

[3-1] Detection of curve area The detection of the curve area will be described.

X calculator 240 ^- control chart calculation unit 242, X ^- use the control chart, the average inside the group from each point of the group

を求める。群の大きさが「２」である場合には、

Ask for. If the group size is “2”,

となる。X⁻管理図計算部２４２は、これに対して、X⁻管理図を作成し、データ蓄積部２３０に格納する。X⁻管理図の作成方法は、例えば、前述の非特許文献４に記載の方法を利用することが可能である。

It becomes. X ^- control chart calculation unit 242, whereas, X ^- to create a control chart is stored in the data storage unit 230. As a method for creating the X ^- control chart, for example, the method described in Non-Patent Document 4 described above can be used.

  FIG. 10 is a flowchart of the curve area detection in the second embodiment of the present invention.

X ^- control chart calculation unit 242

の値が６点（シューハートの管理図による）以上連続して増加（減少）している区間があるかを判定する（ステップ３０１）。ここでなければ（ステップ３０１、No）、「曲線領域は無し」として変化点の検出の処理（ステップ４０１）に移行する。ある場合（ステップ３０１、Yes）は、それらの点を曲線構成点とする。また、この領域を曲線領域とし、データ蓄積部２３０に格納する（ステップ３０２）。

It is determined whether or not there is a section in which the value of is continuously increasing (decreasing) by 6 points or more (according to the Schuhart control chart) (step 301). If not (No in step 301), the process proceeds to the process of detecting the change point (step 401) as “no curve area”. If there is any (step 301, Yes), these points are set as curve constituent points. Further, this region is set as a curve region and stored in the data storage unit 230 (step 302).

  An increase in the sum of the angles formed indicates that the direction is gradually changing, and the original data is a curve. It is determined whether there is a point (extended curve constituent point) that is increased (decreased) together with the curved region by removing a single point that is not continuous in the unclassified point region adjacent to the curved region (step 303). If there is not (step 303, No), the curve area is only the detected curve area, and the process proceeds to step 401. If there is (step 303, Yes), the extended curve composing points are stored in the data storage unit 230 as a curved region, and the process proceeds to step 401 (step 304). Note that a continuous numerical value of 6 points or more may be changed to another numerical value depending on the nature of the data. If the value is small, a slight continuous increase or decrease will be detected as a curved area, and if it is too large, the increase or decrease area will not be detected to some extent. Therefore, the data is changed according to the purpose of data and change point extraction based on 6 points.

[3-2] Change Point Detection Next, change point detection will be described.

  FIG. 11 is a flowchart of change point detection according to the second embodiment of the present invention.

  The change point is detected by the R control chart calculation unit 241.

The R control chart calculation unit 241 obtains a range R _{k obtained} by subtracting the minimum value from the maximum value inside the group from each point of the group read from the data storage unit 230. If the group size is “2”,

となる。連続する未分類点を各未分類点領域としてR管理図を作成し、データ蓄積部２３０に格納する(ステップ４０１）。R管理図の作成方法は、例えば、X⁻管理図と同様に、非特許文献４の技術を利用することが可能である。各R管理図において、R管理限界外の点があるかを判定する（ステップ４０２）。無い場合（ステップ４０２、No）は変化点が存在しないため、ステップ５０１に移行する。ある場合（ステップ４０２、Yes）は、このR管理限界外の点をR変化点とする（ステップ４０３）。

It becomes. An R control chart is created using consecutive unclassified points as respective unclassified point regions, and stored in the data storage unit 230 (step 401). As a method for creating an R control chart, for example, the technique of Non-Patent Document 4 can be used similarly to the X ^- control chart. In each R chart, it is determined whether there is a point outside the R management limit (step 402). If there is no change (step 402, No), the process proceeds to step 501 because there is no change point. If there is any (Step 402, Yes), a point outside the R management limit is set as an R change point (Step 403).

  Next, it is determined whether there is a point within the single R management limit sandwiched between the R change points (step 404). If there is not (step 404, No), each successive uncategorized area is read from the data storage unit 230, and an R control chart having a group size “2” is created for each area. Repeat the process. In some cases (step 404, Yes), an R control chart with a group size of “2” is created for each region of consecutive unclassified points, with points within the R control limit sandwiched between them as R change points. Then, the above process is repeated (step 405).

[3-3] Detection of outliers Next, detection of outliers will be described.

  Detection of outliers is performed by the R control chart calculation unit 241.

  FIG. 12 is a flowchart of outlier detection in the second embodiment of the present invention.

  The R management chart calculation unit 241 sets two consecutive R change points as outlier candidate points in the R management chart read from the data storage unit 230 (step 501). It is determined whether there is an outlier candidate point (step 502). If not (No in Step 502), the process proceeds to Step 601. If there is (Yes in Step 502), the earliest candidate point in time is selected (Step 503). The two unclassified point regions sandwiching the selected outlier candidate points are regarded as one unclassified point region, and an R control chart (temporary connection of unclassified point regions) is created (step 504). However, the outlier candidate points are not used when creating this R chart. It is determined whether all connected unclassified point regions are within the R control limit (confirmation of connection of unclassified point regions) (step 505). If there is at least one point outside the R control limit and the unclassified point area cannot be connected (No in step 505), the two unclassified point areas are not connected and the candidate point is the R change point (Step 506). When all are within the R control limit and the unclassified point regions can be connected (Yes in Step 505), the two unclassified point regions are connected, and the outlier candidate points are set as outliers (Step 507). If it is not determined whether or not there is a missing candidate point (No in step 502), the process proceeds to step 601, and if there is, the next candidate point that is the earliest in time is selected (step 503), and the same is repeated.

[3-4] Detection of linear region Finally, detection of a linear region will be described.

The detection of the straight line area is performed by the X ^- control chart calculation unit 242.

  FIG. 13 is a flowchart of straight line area detection according to the second embodiment of the present invention.

The X ^- control chart calculation unit 242 creates an X ^- control chart for each unclassified point region by using consecutive unclassified points read from the data storage unit 230 as unclassified point regions, and stores them in the data storage unit 230. (Step 601). It is determined whether there is a point outside the control limit in each X ^- control chart read from the data storage unit 230 (step 602). If there is a point outside the control limits (step 602, Yes), X ^- a point outside the control limits

変化点（以下、X⁻変化点と記す）とする。X⁻変化点によって未分類領域が分割され、各未分類領域に対してX⁻管理図を作成し、データ蓄積部２３０に格納する（ステップ６０３）。各X⁻管理図で管理限界外の点がなくなるまで上記の処理を繰り返す。

Let it be a change point (hereinafter referred to as X ^- change point). The unclassified area is divided by X ^- change points, and an X ^- control chart is created for each unclassified area and stored in the data storage unit 230 (step 603). The above process is repeated until there are no points outside the control limits in each X ^- control chart.

  If there is no point outside the control limit (step 602, No), the unclassified point is regarded as a straight component point, the classified point area is defined as a straight line area (step 604), and a straight line is drawn for each straight line area (step 605).

[4] Original data, vector, angle formed, sum of angle formed, group relationship Next, original data (t _k , d _k ) (k = 1,2, ..., n), vector v _k (k = 2 , 3, ..., n), the angle θ _k (k = 3,4,…, n), the sum of the angles

群g_k(k=4,5,…n)との関係を整理する。

The relationship with the group g _k (k = 4, 5,... N) is arranged.

These relationships will be described with reference to FIG. A vector v _k-2 is created from the original data (t _k−3 , d _k−3 ), (t _k−2 , d _k−3 ), and an angle θ _k− formed by v _k−2 and v _{k−1. 1} is made and the resulting angle sum

が作られ、なす角の和

Is the sum of the corners

と

When

から群g_kが作られることを表現している。図１４は、幾何学的にこれらの関係を示したものである。

Expresses that the group g _k is created from. FIG. 14 shows these relationships geometrically.

When these figures are referred to up to a group g _k or from g _k to a curved region, it can be seen that in the original data, up to d _k is a curved region, and from d _{k + 1} is a curved region.

Similarly, until the group g _k, or g in the case of _k was linear region it is understood that to each d _k of the original data is linear region, from d _{k + 1} is a linear region.

Similarly, for change points, up to a certain group g _k , or when there are a plurality of change points from g _k , in the original data, up to d _k is the change point region, and from d _{k + 1} is the change point region.

  The case where the change point is composed of one data (becomes a polygonal line) will be described below.

  In order to explain the procedure for configuring the change point with one data, it is necessary to describe how to draw a line in the straight line region and the curved region.

To draw a straight line in a straight line area,
d _k = bt _k + c (25)
For example, parameters b and c are obtained by the method of least squares.

  How to draw a line in the curve area

というように、例えば2次式で近似し、最小二乗法等によりパラメータa，b，cを求める。

Thus, for example, approximation is performed using a quadratic expression, and parameters a, b, and c are obtained by a least square method or the like.

First, a procedure for configuring a change point sandwiched between two linear regions with one data will be described. When the intersection of the straight lines in the two straight line areas is between the minimum value and the maximum value of the plurality of change points t _k , the intersection is set as the change point. If the intersection of two straight line areas is not between the minimum and maximum values of t _k at multiple change points, it is the largest straight line area on the straight line drawn on the left of the multiple change points. The straight line value at t _k and the straight line value at the smallest t _k in the straight line region on the straight line drawn on the right side of the plurality of change points are connected by a straight line.

If the area between multiple change points is not a straight line area but a curved area, if the point of the curve area adjacent to the multiple change points is (t _k , d _k ), the next straight line

を引く。直線領域で複数の変化点が挟まれる場合と同様に交点が複数の変化点のt_kの最小値と最大値の間にある場合は、その交点を変化点とする。最小値と最大値の間にない場合には、複数の変化点の左側になる曲線領域で引いた曲線上で曲線領域でも最も大きいt_kの時の曲線の値と複数の変化点の右側にある直線領域で引いた曲線上で曲線領域でも最も小さいt_kの時の曲線の値とを直線で結ぶ。

pull. As in the case where a plurality of change points are sandwiched in the straight line region, when the intersection is between the minimum value and the maximum value of t _k of the plurality of change points, the intersection is set as the change point. If it is not between the minimum and maximum values, the curve value at the largest t _k in the curve area on the curve area to the left of the multiple change points and the right side of the multiple change points On the curve drawn in a certain straight line area, the curve value at the smallest t _k in the curved area is connected with a straight line.

  When a plurality of change points are sandwiched between a straight line region and a curved region, a line is drawn in the procedure described so far in each region, and the handling of the intersection point is as described above.

  In data analysis, there is a situation in which data is sequentially input especially in the case of time-series data. Every time new data is acquired, all the procedures described so far may be performed. However, in some cases, the amount of calculation can be reduced.

When the most recent point is in the X ^- management area or in the R management area, the determination may be made by adding a new point to the area, and it is not necessary to determine all the points again.

  However, if the most recent point is other than that, the determination is performed again for all points.

As described above, according to the present embodiment, parameters have already been set in the Schuart X ^-- R control chart itself that is used in quality control for determination, so it is necessary to set parameters as in the prior art. There is no. The amount of calculation treats adjacent data as vectors, finds the vector cross product, finds the sum of the angles formed, and calculates the average and range of the group in the X ⁻ R control chart (summarizing some sum of angle sums) Since only the (range) and the average thereof are obtained, the calculation amount is small. In addition, when new data is obtained, it is only necessary to determine whether the new data is within the control limits of the X ^-- R control chart. Therefore, it is possible to determine whether the point is within the straight line area without new calculation. It is.

  The result output unit 260 outputs the change point, the curve area, the straight line area, and the off-point obtained by the above processing.

  In addition, the operation of the data analysis device 100 shown in FIG. 1 and the data analysis device 200 shown in FIG. 4 is constructed as a program, and the computer is used as a device that detects outliers, change points, curved regions, and straight regions. It can be installed and executed, or distributed via a network.

  The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.

DESCRIPTION OF SYMBOLS 100 Data analyzer 110 Data input part 120 Pre-processing part 130 Vector preparation part 140 Angle creation part 150 Data accumulation part 160 Statistical value calculation part 170 Parameter input part 180 Confidence interval judgment part 190 Result output part 200 Data analysis apparatus 210 Data input part 220 control chart data creating unit 221 vector generating section 222 angle sum creation section 223 group creation unit 230 data storage unit 240 calculating unit 241 R control chart calculator 242 X of ^- control chart calculation unit 250 closed curve division section 260 the result output section

Claims (10)

A data analysis device for analyzing a plurality of two-dimensional data having an order relationship,
A vector-formed angle creating means for creating a vector from adjacent two-dimensional data in the order relationship of the plurality of two-dimensional data, and calculating an angle formed by the adjacent vectors using a vector outer product;
A statistical value calculating means for obtaining an average of the angle formed and a standard deviation of the angle formed;
A confidence interval is calculated from the average of the angles formed and the standard deviation of the angles formed, and depending on whether or not each formed angle is included in the confidence interval, the two-dimensional data that is the calculation source of the angle formed is an outlier or Confidence interval determination means for determining whether or not a change point candidate;
A data analysis apparatus comprising: A data analysis device for analyzing a plurality of two-dimensional data having an order relationship,
A vector-formed angle creating means for creating a vector from adjacent two-dimensional data in the order relationship of the plurality of two-dimensional data, and calculating an angle formed by the adjacent vectors using a vector outer product;
An accumulated value calculation means for calculating an accumulated angle of the angle formed by the vector for each data;
A control chart calculation means for creating a group k that collects a predetermined number of cumulative values of angles formed, and generating a Shuhart X ^-- R control chart from the group k ;
Detecting means for determining whether the two-dimensional data from which the angle formed by the vector is calculated is an outlier, a change point candidate, a straight line region, or a curved region, using a cumulative value of the angle formed by the vector; ,
I have a,
The detection means includes
Wherein X ^- is calculated control limits of -R control chart, the average within each group, range the X ^- depending on whether contained within the control limits of -R control chart, with the calculated origin of the angle Including outlier / change point determination means for determining that certain data is an outlier or a change point,
A data analyzer characterized by that. The confidence interval determination means includes:
The average of the angles formed at N (≧ 3) points including the outlier is calculated, the confidence interval of the average is calculated, and if the average is not included in the confidence interval of the average, it is determined as a change point The data analysis apparatus according to claim 1, further comprising means. The confidence interval determination means includes:
When the standard deviation of the angle formed by the statistical value calculation means is large, it is determined that the given data is the outlier and the change point candidate after performing smoothing by moving average on the given data The data analysis apparatus according to claim 1, further comprising: The detection means includes
Calculating an average of the inside of said group k from each point of the group k, wherein X ^- managed in time series of individual measurements of -R control chart X ^- control charts X ^- value is predetermined point If there is a section in which increased or decreased number of consecutive units or more, X ^- point was a curvilinear configuration point, and the area between the curve area, the points alone is not continuous unclassified dot region adjacent to the curve area 3. The data analysis according to claim 2 , further comprising: a curve area detecting unit that, when there is an extended curve composing point that is increased or decreased together with the curved area by removing the curvilinear area, includes the extended curve composing point. apparatus. The detection means includes
Using the R control chart managed according to the data range of the X ^-- R control chart, a range R obtained by subtracting the minimum value from the maximum value inside the group is obtained from each point of the group, and is outside the R control limit. If there is a point within the single R control limit that is sandwiched between the R change points, a change point detection means that includes the points within the R control limit is included. The data analysis apparatus according to claim 2 . The detecting means is arranged after the curved area detecting means.
Wherein X ^- using control chart, the X ^{- -} X that manages the time series of individual measurements of -R control chart to determine whether there is a point outside the control limits in control charts, when there is, X ^- 6. The data according to claim 5 , further comprising: a straight line area detecting means having a point outside the control limit as an X ^- change point, and if there is no point outside the control limit, an unclassified point as a straight component point and an unclassified point area as a straight line area. Analysis equipment. A data analysis method for analyzing a plurality of two- dimensional data having an order relationship,
Data analysis equipment
Creating a vector from adjacent two-dimensional data in the order relationship of the plurality of two-dimensional data, and calculating an angle formed by the adjacent vectors using a vector outer product;
A statistical value calculating step for obtaining an average of the formed angles and a standard deviation of the formed angles;
A confidence interval is calculated from the average of the angles formed and the standard deviation of the angles formed, and depending on whether or not each formed angle is included in the confidence interval, the two-dimensional data that is the calculation source of the angle formed is an outlier or A confidence interval determination step for determining whether the candidate is a change point;
The data analysis method characterized by performing. A data analysis method for analyzing a plurality of two-dimensional data having an order relationship,
Data analysis equipment
Creating a vector from adjacent two-dimensional data in the order relationship of the plurality of two-dimensional data, and calculating an angle formed by the adjacent vectors using a vector outer product;
A step of calculating a cumulative value of corners for calculating a cumulative value of corners of the vector for each data;
A control chart calculation step of creating a group k that collects a predetermined number of cumulative values of angles formed, and generating a Shuhart X ^-- R control chart from the group k ;
A detection step of determining whether the two-dimensional data that is a calculation source of the angle formed by the vector is an outlier, a change point candidate, a straight line region, or a curved region, using a cumulative value of the angle formed by the vector; ,
The execution,
The detecting step includes
Wherein X ^- is calculated control limits of -R control chart, the average within each group, range the X ^- depending on whether contained within the control limits of -R control chart, with the calculated origin of the angle Determine that some data is an outlier or change point,
A data analysis method characterized by that. Computer
A data analysis program for functioning as each unit of the data analysis apparatus according to any one of claims 1 to 7 .

Images