JP2020027540A - Labeling device, labeling method and program - Google Patents

Abstract

To perform efficient labeling on time-series data.SOLUTION: A labeling device includes: input means for inputting time-series data {x} to be labeled; labeling means for labeling one or more data xincluded in the time series data {x} using one or more time series prediction algorithms selected by a user, out of a plurality of time series prediction algorithms; and output means for outputting the labeled time-series data {x}.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a labeling device, a labeling method, and a program.

  In recent years, with the development of data collection technology, it has become easier to collect data that retains time series (hereinafter, referred to as “time series data”). For this reason, by analyzing the time-series data, it is possible to grasp temporal trends such as trend changes from the past, seasonality, and periodicity. For example, it analyzes time-series sensor data collected from various sensors installed in data centers to predict the occurrence of abnormalities in various devices, and analyzes daily sales data of stores to predict the transition of customers. And so on. In such time-series data analysis, the occurrence of abnormalities and changes in customer withdrawal are predicted using a machine learning method.

  Here, when some prediction is performed using a machine learning method, model learning is often performed using a supervised learning method. In this case, correct label data corresponding to the time-series data is required for learning and evaluating the model. For example, Patent Literature 1 discloses a technique of learning a behavior model by a supervised learning method using time-series sensor data.

  The correct label data may be collected by a sensor or the like, for example, or may be manually created while checking the time-series data. Manually creating correct label data (hereinafter, also simply referred to as “label data”) is referred to as labeling. Such labeling is often performed using, for example, a tool (software) for labeling time-series data.

Japanese Patent No. 5520886

  However, in general, the time-series data has a huge number of data, so that much time is required for manual labeling. Also, in manual labeling, labeling of certain data may be omitted or improper labeling may be performed.

  An embodiment of the present invention has been made in view of the above points, and has as its object to perform efficient labeling on time-series data.

In order to achieve the above object, an embodiment of the present invention provides an input unit for inputting time-series data {x _t } to be labeled, and one or more time-series prediction algorithms selected from a plurality of time-series prediction algorithms. using series prediction algorithm, one or more and labeling means for labeling the data x _t of contained in the time-series data {x _t}, and output means for outputting the time series data {x _t} when the labeling and It is characterized by having.

  Efficient labeling can be performed on time-series data.

It is a figure showing an example of functional composition of a labeling device in an embodiment of the invention. It is a flow chart which shows an example of the flow of labeling processing in an embodiment of the invention. It is a figure (1/2) which shows an example of a labeling screen. It is a figure (2/2) which shows an example of a labeling screen. It is a flow chart which shows an example of a flow of label addition or label deletion in an embodiment of the invention. FIG. 2 is a diagram illustrating an example of a hardware configuration of a labeling device according to an embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described. In the embodiment of the present invention, labeling is performed automatically or manually on time-series data, and information on labeling (for example, information indicating data that is a candidate for labeling, information indicating various statistics, or the like) is provided by a user. , A labeling apparatus 10 that enables efficient labeling will be described. In the embodiment of the present invention, labeling means not only the case where a label is assigned to data included in time-series data, but also the label already assigned to data included in time-series data is deleted. The case is also included. That is, in the embodiment of the present invention, labeling means adding or deleting a label to data included in the time-series data.

  When a label having a predetermined value is given to data, the data may be regarded as “data without a label” (or “data without a label”). For example, if the label value is a binary value of “0” or “1”, the label value “0” represents normal data, and the label value “1” represents abnormal data, a label of “0” is assigned. May be regarded as “data without a label”.

Here, the time-series data is data having a time-series property. The time-series data is represented by, for example, x = {x _t } (t = 1,..., T), where t is an index representing date and time (or may be time or the like). In this case, each x _t may be a data represented by a scalar, may be data that is represented by a vector (i.e., data represented by a plurality of variables). Table for example, when the _{x t} is a vector data represented by three variables _v 1, _{v 2} and _{v 3,} each _{x t is, x t = (v 1,} v 2, v 3) like format Is done. T is an index representing the last date and time when a date and time of a certain time width is considered.

The label data is data indicating a label for data included in the time-series data. The label data is represented by, for example, y = {y _t } (t = 1,..., T). Note that y _t is a label corresponding to the data x _t included in the time-series data. However, the label data may be represented by y = {y _t | y _t {C}, where C is a set of label values assigned to the time-series data.

Note that the time-series data and the label data need not be represented by separate data, but may be represented by one data (that is, labeled time-series data). Each x _t included in the time-series data labeled, it includes labels that are applied to the x _t. For example, if each x _t is a scalar data represented by the variable v, the x _t included in the time-series data with _{labels, x t = (v, y} t) is represented in the form of such. Or, for example, when the _{x t} is a vector data represented by three variables _v 1, _{v 2} and _{v 3,} each _{x t} included in the time-series data with _labels, x t = _(v 1, v ₂ , v ₃ , y _t ).

<Functional configuration of labeling device 10>
First, a functional configuration of a labeling device 10 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of a functional configuration of a labeling device 10 according to an embodiment of the present invention.

  As shown in FIG. 1, the labeling device 10 according to the embodiment of the present invention includes a display control unit 101, an operation reception unit 102, a data input unit 103, a label processing unit 104, a statistic calculation unit 105, It has a label assignment candidate extraction unit 106 and a data output unit 107. These units are realized, for example, by a process in which a labeling program 100 installed in the labeling apparatus 10 causes a CPU (Central Processing Unit) to execute. Further, the labeling device 10 according to the embodiment of the present invention has a storage unit 201. Note that the labeling program 100 may be a single program or may be composed of a plurality of program modules.

  The storage unit 201 is a storage area used by the labeling program 100, and stores various data. Examples of data stored in the storage unit 201 include time-series data and labeling data.

  The display control unit 101 performs various controls related to display. For example, the display control unit 101 displays a labeling screen described later, and displays various information in the labeling screen.

  The operation receiving unit 102 receives input of various operations from a user. For example, the operation receiving unit 102 receives various operations on the labeling screen displayed by the display control unit 101.

  The data input unit 103 inputs time-series data (including time-series data with a label). Here, the data input unit 103 can input time-series data in an arbitrary format according to, for example, a user's selection.

  For example, the data input unit 103 may input a time-series data stored in the file by reading a file stored in an auxiliary storage device or the like of the labeling device 10, or may communicate with the labeling device 10. By reading a file uploaded from a terminal device or the like connected via a network, time-series data stored in this file may be input. Alternatively, for example, the data input unit 103 may input time-series data from a database server or the like by an SQL query described or selected by a user. At this time, the data input unit 103 may input time-series data from a plurality of database servers or the like by, for example, a JOIN operation or the like.

  The label processing unit 104 performs automatic labeling or manual labeling on the time-series data input by the data input unit 103. Here, the manual labeling refers to, for example, labeling data set by a user operation or data belonging to a range set by a user operation. The automatic labeling is, for example, labeling data extracted from the time-series prediction algorithm and parameters using a time-series prediction algorithm and parameters selected by a user operation.

  By using the automatic labeling, the user can select a desired time-series prediction algorithm from a plurality of time-series prediction algorithms and set parameters of the selected time-series algorithm, and can automatically generate the time-series data. Labeling can be performed effectively.

  The statistic calculation unit 105 calculates a predetermined statistic from time-series data, label data, and the like. Here, as the predetermined statistic, for example, a ratio (percentage or the like) between the number of labeled data and the number of unlabeled data in the time-series data is exemplified. In addition, examples of the predetermined statistic include a minimum value, a median value, an average value, and a maximum value of data to which a label is assigned in time-series data. Similarly, for example, the minimum value, the median value, the average value, the maximum value, and the like of the unlabeled data in the time-series data are given.

  Information indicating the statistic calculated by the statistic calculation unit 105 (hereinafter, also referred to as “statistic information”) is displayed by the display control unit 101 in the labeling screen. By confirming the statistic information displayed in the labeling screen, the user can refer to when labeling the time-series data.

  The label assignment candidate extraction unit 106 selects the time-series data input by the data input unit 103 in accordance with the selection of data to which a label is assigned (hereinafter, also referred to as “labeled data”) and the setting of parameters. , Data that is a candidate for label assignment is extracted. The label assignment candidate extraction unit 106, for example, creates a partial series from the time-series data, and then extracts candidate data for label assignment based on the distance from the partial series.

  The data extracted by the label assignment candidate extraction unit 106 is displayed on the labeling screen by the display control unit 101 as label assignment candidate information. By checking the label assignment candidate information displayed in the labeling screen, the user can know the data that is a candidate for label assignment.

  The data output unit 107 outputs time-series data (including labeled time-series data). Here, the data output unit 107 can output time-series data in an arbitrary format according to, for example, a user's selection or the like.

  For example, the data output unit 107 may output time-series data as a file to an auxiliary storage device or the like of the labeling device 10, or may output the time-series data to a server device or the like connected to the labeling device 10 via a communication network. The data may be output as a file. Alternatively, for example, the data output unit 107 may output time-series data to a database server or the like. Note that the data output unit 107 may output only label data instead of time-series data or labeled time-series data.

<Labeling process>
Hereinafter, a labeling process performed by the labeling apparatus 10 according to the embodiment of the present invention will be described with reference to FIG. FIG. 2 is a flowchart illustrating an example of the flow of the labeling process according to the embodiment of the present invention.

  Step S101: First, the display control unit 101 displays a labeling screen G100 for labeling time-series data according to, for example, activation of the labeling program 100 or the like. Here, the labeling screen G100 will be described with reference to FIGS. 3 and 4 are diagrams illustrating an example of the labeling screen G100. Note that the labeling screen G100 shown in FIG. 3 and the labeling screen G100 shown in FIG. 4 can be transitioned to each other by, for example, scrolling the screen using a scroll bar or the like. However, the present invention is not limited thereto, and the labeling screen G100 shown in FIG. 3 and the labeling screen G100 shown in FIG.

  The labeling screen G100 includes a data input column G110, a display variable selection column G120, a time-series data (graph display) column G130, an enlarged display column G140, a manual labeling (range designation) column G150, and an automatic labeling column G160. , A label assignment or label deletion section G170, a label assignment candidate extraction section G180, a data output section G190, a time series data (list display) section G200, a statistic information section G210, and a label assignment candidate information section G220. Is included.

  The data input field G110 is an area for setting and executing an input method when inputting time-series data by the data input unit 103. The data input field G110 includes an input method setting field G111 for setting a data input method, and an input button G112 for executing time-series data input. In the input method setting field G111, the user can set various input methods such as, for example, reading a file and inputting time-series data, and inputting time-series data from a database.

  The display variable selection field G120 is an area for selecting a variable displayed as a graph in the time series data (graph display) field G130. In the display variable selection field G120, the user displays variables as graphs in the time-series data (graph display) field G130 among variables of each data included in the time-series data (hereinafter, also referred to as “display target variables”). Can be selected.

  The time-series data (graph display) column G130 includes a graph (hereinafter, “time”) relating to a certain variable (the display target variable selected in the display variable selection column G120) of the time-series data input by the data input unit 103. A series graph is also displayed.). The enlarged display section G140 is an area for enlarging and displaying the time series graph displayed in the time series data (graph display) section G130.

  The manual labeling (range specification) column G150 is an area for setting the range of date and time when performing manual labeling by specifying the range of date and time by the label processing unit 104. The automatic labeling section G160 is an area for selecting a time-series prediction algorithm and setting parameters when performing automatic labeling by the label processing unit 104. The label addition or label deletion column G170 is an area for setting a label value and performing manual labeling or automatic labeling.

  The label assignment candidate extraction column G180 is an area for setting and executing parameters when the label assignment candidate extraction unit 106 extracts a label assignment candidate.

  The data output column G190 is an area for setting and executing an output method when data (time-series data, labeled time-series data, label data, or the like) is output by the data output unit 107.

  The time-series data (list display) column G200 is an area in which each data included in the time-series data input by the data input unit 103 is displayed as a list. That is, the time series data (list display) column G200 is an area in which each data included in the time series data is displayed as it is as raw data.

  The statistic information column G210 is an area where the statistic calculated by the statistic calculation unit 105 is displayed as statistic information. The label assignment candidate information column G220 is an area where the label assignment candidates extracted by the label assignment candidate extraction unit 106 are displayed as label assignment candidate information.

  As described above, the labeling screen G100 includes a plurality of areas, and various operations, information references, and the like can be performed in these areas. Details of each of these areas will be described later. Note that various operations (for example, an input operation, a selection operation, and a setting operation) of the user on the labeling screen G100 are received by the operation reception unit 102.

  Step S102: Next, the data input unit 103 inputs time-series data (including labeled time-series data) in accordance with a user operation in the data input field G110 of the labeling screen G100. Note that the time-series data input by the data input unit 103 is stored in the storage unit 201.

  The user sets a desired input method in the input method setting field G111 of the data input field G110, and then performs an operation of pressing the input button G112, whereby time-series data can be input by this input method. When setting the input method in the input method setting field G111, the user also sets parameters according to the input method. For example, when “input from database” is set as the input method, an SQL for inputting time-series data from the corresponding database is described or selected. Further, for example, when “input from file” is set as the input method, the storage destination address or the like of the file is specified or selected.

  When the time-series data is input in step S102, the display control unit 101 displays each data included in the time-series data as a list in the time-series data (list display) column G200.

Step S103: Next, the operation receiving unit 102 receives selection of a variable to be displayed on the labeling screen G100. The user can select a display target variable by inputting a variable in the selection column G121 included in the display variable selection column G120. The variable that can be input to the selection field G121 is any one of the variables of each data included in the time-series data input in step S102. For example, when the case where each data _{x t} included in the sequence data is represented as _{= x t (v 1, v} 2, v 3), is enterable variables selection column _G121, v 1, _{v 2} and _{v 3} Is one of the variables.

When the time-series data is labeled time-series data, variables other than the label value can be input to the selection field G121. For example, if each data _{x t} that is included in the time-series data labeled is represented by _{= x t (v 1, v} 2, v 3, y t), it is enterable variables selection column _{G121, v} 1, v is any one variable of ₂ and _{v 3.}

  However, two or more variables may be input to the selection field G121. In this case, in the time series data (graph display) column G130, graphs relating to the plurality of display target variables input to the selection column G121 are displayed.

  When the display target variable is selected in step S103, the display control unit 101 displays a graph relating to the display target variable of the time-series data in the time-series data (graph display) column G130. That is, the time series data (graph display) column G130 includes a graph G131 of all time sections of the time series data and a graph G132 of a certain time section. In the graphs G131 and G132, for example, a mark such as a circle is displayed for the labeled data.

Here, the graph G132 is a graph in which a certain time section (for example, a time section specified by the user) of the graph G131 is enlarged and displayed. The user inputs a desired start date and time in the start date and time input field G141 included in the enlarged display field G140, inputs a desired end date and time in the end date and time input field G142, and presses the display button G143. The time series data of the time section from the input start date and time to the end time can be displayed as a graph G132. However, start date / time <end date / time. For example, the time-series data _x = _{x t} input in step S102 described above (t = 1, ···, T ) if it is, graph G131 _is, x 1, · · ·, adjacent _{x T} 6 is a graph in which data to be connected are connected by line segments. Meanwhile, the start time ts (≧ 1), if the end time was te (≦ T), graph G132 _is, x ts, · · _·, a graph are connected by line segments adjacent data between the _{x te} is there. Note that the user can also display the time series data of the time section as a graph G132 by, for example, specifying a range of a desired time section from the graph G131.

  The following steps S104 to S107 are executed in response to a user operation after a variable to be displayed is selected. That is, when the user performs an operation for performing manual labeling or an operation for performing automatic labeling, Steps S104 to S105 are performed, and when the user performs an operation for extracting a label assignment candidate, Step S104 to Step S105 are performed. Steps S106 to S107 are performed. Steps S104 to S107 can be repeatedly executed an arbitrary number of times until an operation for outputting time-series data after labeling is performed.

  Step S104: An operation for performing manual labeling (hereinafter, referred to as “manual labeling operation”) or an operation for performing automatic labeling (hereinafter, referred to as “automatic labeling operation”) is performed by the user. In this case, the operation receiving unit 102 receives the operation. Then, the label processing unit 104 performs label addition or label deletion by manual labeling or automatic labeling.

  Here, the user can perform a manual labeling operation or an automatic labeling operation by the following operations.

(Manual labeling operation)
The user inputs a desired start date and time in the start date and time input column G151 included in the manual labeling (range designation) column G150, and inputs a desired end date and time in the end date and time input column G152. However, start date / time <end date / time. Then, when performing label assignment, the user selects “manual” in the manual / automatic sorting selection field G171 included in the label assignment or label deletion field G170, and inputs a desired label value in the label input field G172. Then, the labeling button G173 is pressed. On the other hand, when performing label deletion, the user selects “manual” in the manual / automatic classification selection field G171 included in the label assignment or label deletion field G170, and then presses the delete button G174.

  This allows the user to perform a manual labeling operation. By performing the manual labeling operation, labeling of data included in the time section from the start date and time to the end date and time among the data included in the time-series data is performed. That is, in the case of labeling, the label of the label value is added to the data included in the time section. On the other hand, in the case of label deletion, the label is deleted from the data included in the time section.

  Note that the user may be able to perform a manual labeling operation by selecting, for example, data in the graph G132 displayed in the time-series data (graph display) column G130 with a pointing device such as a mouse. At this time, in the case of labeling, for example, after inputting a desired label value in the label input field G172, the user may select desired data in the graph G132 using a pointing device. On the other hand, in the case of label deletion, for example, the user selects desired data in the graph G132 with a pointing device in a state where nothing is input in the label input field G172 (blank state). good. Alternatively, label deletion may be performed by, for example, double-clicking desired data in the graph G132 with a pointing device.

(Automatic labeling operation)
The user selects a desired time-series prediction algorithm (or a combination of time-series algorithms) from the algorithm selection field G161 included in the automatic labeling field G160, and selects a desired time-series prediction algorithm (or a combination of time-series algorithms). The parameters required for data extraction are set in the parameter setting column G162. Then, when performing label assignment, the user selects "automatic" in the manual / automatic sorting selection field G171 included in the label assignment or label deletion field G170, and inputs a desired label value in the label input field G172. Then, the labeling button G173 is pressed. On the other hand, when performing label deletion, the user selects “automatic” in the manual / automatic classification selection field G171 included in the label assignment or label deletion field G170, and then presses the delete button G174.

  Thereby, the user can perform an automatic labeling operation. By performing the automatic labeling operation, the data is extracted by the time-series prediction algorithm (or a combination of the time-series algorithms) selected by the user, and the extracted data is extracted from the data included in the time-series data. Labeled. The time series prediction algorithm selectable in the algorithm selection field G161 includes, for example, Prophet, ARIMA, Singular Spectrum Transformation, Moving Average, and the like. However, the time-series prediction algorithm selectable in the algorithm selection field G161 is not limited to these, and any algorithm can be selected in the algorithm selection field G161 as long as it is an algorithm that can predict time-series data.

  In the manual labeling operation and the automatic labeling operation described above, the user selects either “manual” or “automatic” in the manual / automatic classification selection field G171, but this selection operation may be unnecessary. In this case, for example, when the user presses the labeling button G173 or the delete button G174, the label processing unit 104 may determine whether the operation is “manual” or “automatic”. At this time, for example, when the start date and time and the end date and time are respectively set in the start date and time input column G151 and the end date and time input column G152 included in the manual labeling (range designation) column G150, “manual” is determined. When the time-series prediction algorithm is selected in the algorithm selection field G161 and a parameter is set in the parameter setting field G162, it may be determined to be "automatic".

  Here, the flow of processing for label addition or label deletion (that is, labeling) in step S104 will be described with reference to FIG. FIG. 5 is a flowchart illustrating an example of the flow of label addition or label deletion in the embodiment of the present invention. This processing is executed when the operation receiving unit 102 receives either the manual labeling operation or the automatic labeling operation.

  Step S201: First, the label processing unit 104 determines whether the operation received by the operation receiving unit 102 is a manual labeling operation or an automatic labeling operation. If the result of this determination is a manual labeling operation, the label processing unit 104 proceeds to step S202. On the other hand, if the result of this determination is an automatic labeling operation, the label processing unit 104 proceeds to step S203.

Step S202: The label processing unit 104 performs label assignment or label deletion on the data selected by the manual labeling operation. That is, in the case of labeling, the label processing unit 104 performs processing on data included in a time section between the start date and time input in the start date and time input field G151 and the end date and time input in the end date and time input field G152. , The input label value is assigned to the label input field G172. Thus, the label y _t is given to the labeling target data x _{t (i.e.,} data x _t manually select labeling _operation). Note that this label _{y t} is stored in the storage unit 201.

On the other hand, in the case of label deletion, the label processing unit 104 adds the data included in the time section from the start date and time input in the start date and time input field G151 to the end date and time input in the end date and time input field G152. Remove the label that is being used. Thus, labeling target data x _{t (i.e.,} manual data selected by the labeling operation x _t) label y _t is removed from.

  Note that deleting the label may be updating or overwriting the label value to a predetermined value. For example, if the label value is a binary value “0” or “1”, the label value “0” represents normal data, and the label value “1” represents abnormal data, the deletion of the label is The label value of the data to which the label of “1” is assigned may be updated or overwritten to “0”.

  Step S203: The label processing unit 104 is required to extract the time-series prediction algorithm (or a combination of the time-series algorithms) selected by the automatic labeling operation and to extract data with the time-series prediction algorithm (or the combination of the time-series algorithms). The data to be labeled is extracted based on the parameters.

  Here, an example will be described in which the label processing unit 104 extracts data to be labeled when a time-series prediction algorithm that outputs a time-series predicted value and a confidence interval is selected. In addition, as a time-series prediction algorithm for outputting a time-series prediction value and a confidence interval, for example, Prophet or the like is given.

In this case, the time-series prediction algorithm receives the time-series data x = {x _t } (t = 1,..., T) as input, and the time-series data {x _t } (t = 1,. ..., and T), and outputs the confidence interval _{{u t}} and _{{l t}.} Here, _{u t} and _{l t} are the upper and lower limits of the confidence interval for each estimated value ^ _{x t.} In addition, for convenience, a hat “＾” indicating a predicted value is described immediately before a character, not above the character.

At this time, the label processing unit 104, for _example, x 1, among · · · _{x T,} may be extracted x t> _{u t} or _l t> _{x x t} that satisfies any of _t as labeling target data . In this case, the user does not need to set parameters.

However, the present invention is not limited to this. For example, when a threshold θ is set by a user as a parameter of the time-series prediction algorithm, data to be labeled may be extracted using the threshold. For _example, it may be extracted x _t -u t> θ or _{l t} -x t> _x t that satisfies any of theta as labeling target data. Or, for _{example, | x t - ^ x t} |> θ may be extracted _{x t} as labeling target data satisfying. At this time, a weight may be set as a parameter. For example, <if (a ≦ 1 is set, _{| x} t -a × _{^ x} t | weight a 0)> θ may be to extract the _{x t} as labeling target data satisfying.

When a combination of the time-series prediction algorithms is selected by the user, data to be labeled is extracted from the plurality of time-series prediction algorithms. For example, when the combination of the first time-series prediction algorithm and the second time-series prediction algorithm is selected, the time-series data {x ′ _t } (t = 1) of the predicted value by the first time-series prediction algorithm ,..., T) and the time-series data {x ″ _t } (t = 1,..., T) of the prediction value by the second time-series prediction algorithm, | x _t − _{(a × ^ x't + b} × ^ x'' t) |> θ may be extracted _{x t} as labeling target data satisfying. Here, a, b, and θ are parameters set by the user, and a and b are weights of the first time series prediction algorithm and the second time series prediction algorithm, for example, a + b = 1, a ≧ 0, b ≧ 0.

In addition to the above, for example, when the first time-series prediction algorithm and the second time-series prediction algorithm output a confidence interval, the confidence interval output by the first time-series prediction algorithm is set to ｛u ′ _t ｝ And {l ′ _t }, and the confidence intervals output by the second time-series prediction algorithm are defined as {u ″ _t } and {l ″ _t }, and x _t > (u ′ _t + u ″ _t ) / 2. or _{(l't + l'' t) /} 2> x t that satisfies any of the _{x t} may be extracted as the labeling target data. Further, the combination of the time-series prediction algorithms is not limited to two, and may be a combination of three or more time-series prediction algorithms.

In the above description, the case of extracting data to be labeled using the time-series prediction algorithm has been described. For example, the threshold θ set by the user and the time-series data x = {x _t } (t = 1, .., T), the data to be labeled may be extracted. For example, to extract the x _t satisfying x _t> theta as labeling target data (or, to extract the x _t satisfying x t _<theta as labeling target data) may be like.

Further, the condition for extracting the data to be labeled may be changed according to the label value set in the label input field G172. For example, in the case of using the time-series prediction algorithm above confidence interval is output, it is determined whether or not satisfy x _t> u _t when the label value is "0", the label value is "1" At some point, it is determined whether or not l _t > x _t is satisfied. Alternatively, for example, when comparing a predetermined threshold value with the time-series data x = {x _t }, if the label value is “0”, the threshold value θ ₀ is used and x _t > θ ₀ is set. It determines whether they meet, when the label value is "1", using a threshold theta _1, determines whether or not satisfy x _t> theta _1, and so on. Step S204: The label processing unit 104 performs label assignment or label deletion on the data extracted in step S203. That is, in the case of label assignment, the label processing unit 104 assigns the label value input to the label input field G172 to the data extracted in step S203. Thus, the label _{y t} is given to the labeling target data _{x t} (i.e., the data _{x t} extracted in step S203 above). Note that this label _{y t} is stored in the storage unit 201.

On the other hand, in the case of label deletion, the label processing unit 104 deletes the label given to the data extracted in step S203. Thus, labeling target data _{x t} (i.e., the extracted data _{x t} at step S203 described above) label _{y t} from is deleted.

  It should be noted that, when performing labeling as described above, there may be cases where a label has already been allocated to the data extracted in step S203. In this case, the already assigned label may be updated or overwritten. That is, the pressing operation of the labeling button G173 may mean not only the operation of giving the label but also the operation of updating or overwriting the label. Further, in addition to updating or overwriting the label, a label may be additionally provided to the data extracted in step S203. Further, data to which a label has already been assigned may not be extracted in step S203. In this case, (1) update or overwrite a label already assigned, (2) add a label additionally, (3) prevent data already labeled from being extracted, Which of (1) to (3) above may be selected by the user, for example.

In addition, when performing the label assignment as described above, the label value set in the label entry field G172 is assigned to the data extracted in step S203. However, the present invention is not limited thereto. The operation of inputting the label value to is unnecessary, and the label value may be assigned according to the data extracted in step S203. In this case, for example, in the time-series prediction algorithm reliability interval it is output, it is conceivable to impart a label value corresponding to the value of x _t -u _t values or l _t -x _t on x _t. Specifically, for example, as δ _{t =} x _t -u _t, label value when [delta] _t is within a range "1", if in another a range as such label value "2" Next, it is conceivable to determine a label value to be assigned to data. Similarly, for example, it is conceivable to impart a label value corresponding to the difference between the even θ and x _t when determining the label value by comparison with the set threshold θ and x _t by the user x _t.

  As described above, deleting a label may be updating or overwriting a label value to a predetermined value. For example, if the label value is a binary value “0” or “1”, the label value “0” represents normal data, and the label value “1” represents abnormal data, the deletion of the label is The label value of the data to which the label of “1” is assigned may be updated or overwritten to “0”.

  Return to FIG. Step S105: The statistic calculation unit 105 calculates a predetermined statistic from time-series data, label data, and the like. Then, the display control unit 101 updates the statistic information displayed in the statistic information column G210 with the statistic calculated by the statistic calculation unit 105.

  Here, the statistic information column G210 shown in FIG. 4 includes, in the time series data, the number of data to which the label with the label value “0” is assigned and the data to which the label with the label value “1” is assigned. The ratio with the number is displayed as a pie chart, but this is an example, and an arbitrary statistic calculated by the statistic calculation unit 105 is displayed in an arbitrary format in the statistic information column G210. . For example, as a pie chart, the number of data items labeled with a label value “0”, the number of data items labeled with a label value “1”, and the number of data items without a label (in this case, The data with no label is data without the label value itself.). In addition to the pie chart, for example, the statistic calculated by the statistic calculation unit 105 is the minimum value, the median value, the average value, and the maximum value of the labeled data and the unlabeled data. In the case of a value or the like, the statistic information column G210 may display a boxplot for data with a label and a boxplot for data without a label.

  Step S106: When an operation for extracting a label assignment candidate (hereinafter, also referred to as “label assignment candidate extraction operation”) is performed by the user, the operation accepting unit 102 accepts the operation. Then, the label assignment candidate extracting unit 106 extracts data that is a candidate for label assignment from the time-series data input by the data input unit 103.

  Here, the user sets the parameters necessary for extracting the label assignment candidate in the label assignment candidate extraction column G180, and selects reference data (hereinafter, referred to as “reference data”) from the graph G132. Then, by pressing the extraction button G185, a label assignment candidate extraction operation can be performed. These parameters can be set, for example, in a distance selection column G181, a target point selection column G182, a time length setting column G183, and an output number setting column G184. Also, as the reference data, one data is selected from the data to which the label has been added.

  In the distance selection field G181, a distance used for extracting a label assignment candidate is selected. Examples of the distance that can be selected in the distance selection column G181 include DTW (Dynamic Time Warping) and Euclidean distance. Other than these, for example, a Fourier transform distance, an autoregressive coefficient distance, or the like may be selectable.

  In the target point selection column G182, a reference position in the partial sequence is selected when a partial sequence having the time length set in the time length setting column G183 is created. Reference positions that can be set in the target point selection field G182 include “First”, “Center”, “Last”, and the like.

The time length indicating the length of the partial sequence is set in the time length setting column G183. Here, in the embodiment of the present invention, it is assumed that the time length is represented by the length of the index of the time-series data x = {x _t }. However, the time length is not limited to the length of the index, and may be represented by, for example, a time width or the like.

  In the output number setting column G184, the number of data extracted by the label assignment candidate extraction unit 106 is set.

Here, a case will be described in which a label assignment candidate is extracted from the time-series data x = {x _t } (t = 1,..., T) using the above parameters and the reference data x _{t ′.} . Note that, as an example, among the above parameters, the time length is L and the number of outputs is N.

First, the label assignment candidate extraction unit 106 creates a subsequence having a time length L including the reference data _{xt '} at the reference position. For example, when the reference position is “First”, the label assignment candidate extraction unit 106 creates a subsequence having a time length L in which the reference data _{xt ′} is at the head. Further, for example, when the reference position is “Center”, the label assignment candidate extraction unit 106 determines that the sub-sequence of the time length L centered on the reference data _{xt ′} (that is, for example, L / 2 (L: even number)) A (L + 1) / 2 (L: odd) -th reference sequence _{xt '} with a time length L is created. Further, for example, when the reference position is “Last”, the label assignment candidate extraction unit 106 creates a subsequence having a time length L ending with the reference data _{xt ′} . Hereinafter, the partial sequence including the reference data _{xt '} is referred to as "reference partial sequence".

Next, the label assignment candidate extracting unit 106 includes a sub-sequence having a time length L (hereinafter, referred to as a “target sub-sequence”) that includes unlabeled data in the time-series data at the reference position, and a reference sub-sequence. Calculate the distance between For example, assuming that a set of date and time t of unlabeled data in the time-series data is S, for example, a target subsequence including unlabeled data at the reference position “First” is represented by t∈S, x _t, ···, represented by _{x t + L-1.} Similarly, for example, the target partial sequence containing data that is not the label applied to the reference position "Last" as _{t∈S, x t-L + 1} , ···, represented by _{x t.} Therefore, the label assignment candidate extraction unit 106 calculates the distance between these target subsequences and the reference subsequences for each t∈S.

  Then, the label assignment candidate extraction unit 106 specifies the top N target subsequences in ascending order of distance from the reference subsequence, and extracts data at the reference position of the specified target subsequence. As a result, data that is a candidate for label assignment is extracted. This is because the data at the reference position of the target subsequence whose distance from the reference subsequence is small can be given the same label as the reference data (or the same label as the reference data is likely to be given). ).

  Step S107: The display control unit 101 updates the label assignment candidate information displayed in the label assignment candidate information column G220 with the data (label assignment candidates) extracted in step S106.

  Here, in the label assignment candidate information column G220 shown in FIG. 4, the top three data extracted in step S106 are displayed as "candidates". The “distance” is the distance between the target partial sequence including the data and the reference partial sequence. The “reference data date and time” and the “reference data label” are the date and time and the label value of the reference data, respectively.

  Note that the display control unit 101 may further display information indicating the data extracted in step S106 in the time-series data (graph display) column G130. For example, in the graph G132, a vertical line may be displayed at the date and time of the extracted data, or an icon or the like in a prominent manner may be displayed. Accordingly, the user can easily and easily know which data on the graph G132 indicates the data indicating the label assignment candidate.

  Step S108: Finally, the data output unit 107 outputs the time-series data after labeling from the storage unit 201 in accordance with a user operation in the data output column G190 of the labeling screen G100.

  The user sets a desired output method in the output method setting field G191 of the data output field G190, and then performs an operation of pressing the output button G192 to input the time-series data after labeling by this output method. Can be. Here, the time-series data after labeling may be labeled time-series data, or may be a set of time-series data and label data. Further, the data output unit 107 may output only the label data.

  When setting the output method in the output method setting column G191, the user also sets parameters according to the output method. For example, when "output to database" is set as the output method, the corresponding database is designated or selected. Further, for example, when “output to file” is set as the output method, the storage destination address of the file is specified or selected.

<Hardware configuration of labeling device 10>
Next, a hardware configuration of the labeling device 10 according to the embodiment of the present invention will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of a hardware configuration of the labeling device 10 according to the embodiment of the present invention.

  As shown in FIG. 6, the labeling device 10 according to the embodiment of the present invention includes an input device 301, a display device 302, an external I / F 303, a RAM (Random Access Memory) 304, and a ROM (Read Only Memory). 305, a CPU 306, a communication I / F 307, and an auxiliary storage device 308. Each of these pieces of hardware is communicably connected via a bus B.

  The input device 301 is, for example, a keyboard, a mouse, a touch panel, or the like, and is used by a user to input various operations. The display device 302 is, for example, a display or the like, and displays a processing result of the labeling device 10.

  The external I / F 303 is an interface with an external device. The external device includes a recording medium 303a and the like. The labeling device 10 can read and write the recording medium 303a and the like via the external I / F 303. The labeling program 100 and the like may be recorded on the recording medium 303a.

  Examples of the recording medium 303a include a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disk), an SD memory card (Secure Digital memory card), and a USB (Universal Serial Bus) memory card.

  The RAM 304 is a volatile semiconductor memory that temporarily stores programs and data. The ROM 305 is a nonvolatile semiconductor memory that can retain programs and data even when the power is turned off. The ROM 305 stores, for example, setting information about an OS (Operating System), setting information about a communication network, and the like.

  The CPU 306 is an arithmetic unit that reads out programs and data from the ROM 305 and the auxiliary storage device 308 onto the RAM 304 and executes various processes.

  The communication I / F 307 is an interface for connecting the labeling device 10 to a communication network. The labeling program 100 may be obtained (downloaded) from a predetermined server or the like via the communication I / F 307.

  The auxiliary storage device 308 is, for example, a hard disk drive (HDD) or a solid state drive (SSD), and is a non-volatile storage device that stores programs and data. The programs and data stored in the auxiliary storage device 308 include, for example, an OS, an application program, a labeling program 100, and the like. Note that the storage unit 201 is realized by, for example, the auxiliary storage device 308 or the like.

  The labeling device 10 according to the embodiment of the present invention has the hardware configuration shown in FIG. 6, so that the various processes described above can be realized. In the example illustrated in FIG. 6, the hardware configuration in the case where the labeling device 10 is realized by one computer has been described. However, the present invention is not limited thereto, and the labeling device 10 may be realized by a plurality of computers. good.

<Summary>
As described above, the labeling device 10 according to the embodiment of the present invention can automatically or manually label (label or delete) time-series data on the labeling screen G100 by a user operation. In particular, the labeling device 10 according to the embodiment of the present invention can automatically label time-series data using a time-series prediction algorithm. Therefore, efficient labeling can be performed on the time-series data. That is, labeling can be performed on time-series data at high speed and with high accuracy.

  Further, the labeling device 10 according to the embodiment of the present invention displays statistic information, label assignment candidate information, and the like in the labeling screen G100. Therefore, the user can refer to these pieces of information to check whether the labeling result for the time-series data is appropriate or not, and to check whether there is no omission of label addition.

  The present invention is not limited to the above-described embodiments specifically disclosed, and various modifications and changes can be made without departing from the scope of the claims.

REFERENCE SIGNS LIST 10 labeling device 100 labeling program 101 display control unit 102 operation receiving unit 103 data input unit 104 label processing unit 105 statistic calculation unit 106 labeling candidate extraction unit 107 data output unit 201 storage unit

Claims (8)

Input means for inputting time-series data {x _t } to be labeled;
Among a plurality of time-series prediction algorithm, and labeling means for using one or more time series prediction algorithm selected by the user, labeling one or more data x _t that is included in the time-series data {x _t},
Output means for outputting the labeled time-series data {x _t };
A labeling device comprising: The labeling means,
The time series data {x _t } is obtained by using one or more prediction series predicted from the time series data {x _t } by the one or more time series prediction algorithms and the time series data {x _t }. and the difference between the weighted sum of the predicted data of the index t in the data x _t and one or more of the predicted sequences included, by comparison with a predetermined threshold value, labeling the data x _t, wherein the The labeling device according to claim 1. The labeling means,
The upper limit value sequence {u _t} and the lower limit value sequence {l _t} confidence interval of the time-series data {x _t} predicted predicted sequence from {^ x _t} by the time-series prediction algorithm, the time-series data by using the _{{x t},} and the data _{x t} that is included in the time-series data _{{x t},} the upper limit value series limit _{u t} or the lower limit value sequence contained in _{{u t} {l t}} the result of comparison between the lower limit value l _t contained, labeling apparatus according to claim 1, wherein the labeling data x _t, it is characterized. The time of data x _t that is included in the series data {x _t}, the data x _t no label is applied, claims 1 to 3 having an extraction means for extracting as a candidate for labeling, characterized in that The labeling device according to any one of claims 1 to 4. The extracting means,
The distance between the first subsequence including the data with the label selected by the user and the second subsequence included in the time-series data {x _t } is calculated, and the calculated distance is in ascending order. The labeling apparatus according to claim 4, wherein data included in a predetermined position of a second predetermined number of second subsequences is extracted as the label assignment candidate. 6. The display device according to claim 4, further comprising a display unit configured to display a screen including at least the label assignment candidate and information indicating a predetermined statistic of the labeled time-series data {x _t }. The labeling device according to item 1. Computer
An input procedure for inputting time-series data {x _t } to be labeled;
Among a plurality of time-series prediction algorithm, and labeling procedure using one or more time series prediction algorithm selected by the user, labeling one or more data x _t that is included in the time-series data {x _t},
An output procedure for outputting the labeled time-series data {x _t };
A labeling method.   A program for causing a computer to function as each unit in the labeling device according to claim 1.

Images