WO2022239245A1

WO2022239245A1 - Training method, inference method, training device, inference device, and program

Info

Publication number: WO2022239245A1
Application number: PCT/JP2021/018484
Authority: WO
Inventors: 雄貴蔵内; 祥章瀧本; 修平山本
Original assignee: 日本電信電話株式会社
Priority date: 2021-05-14
Filing date: 2021-05-14
Publication date: 2022-11-17
Also published as: JPWO2022239245A1

Abstract

A training method according to one embodiment of the present invention is such that a computer executes: an input step for inputting multidimensional data having an estimation object attribute indicating the attribute of an estimation object, and at least two non-estimation object attributes indicating attributes other than the estimation object attribute; a dimension reduction step for reducing the number of dimensions of the non-estimation object attributes in the multidimensional data; a binning step for performing binning on the values of the non-estimation object attributes in the multidimensional data after the dimensional reduction; an information addition step for adding prescribed additional information to the multidimensional data after the binning; and a training step for training a parameter of an inference model for estimating the value of the estimation object attribute by using the multidimensional data having the additional information added thereto.

Description

Learning method, reasoning method, learning device, reasoning device, and program

The present invention relates to a learning method, an inference method, a learning device, an inference device, and a program.

The task of estimating certain specific information using multidimensional data with multiple attributes such as date, gender, age, etc., is widely performed.

Known methods for estimating information using multidimensional data include, for example, multi-layer perceptron (MLP) and N-dimensional convolution (see Non-Patent Document 1).

In the above MLP, inference is performed using only attribute value sets. For example, specific information (for example, the number of contracts) is estimated by inputting only an attribute value set such as (year/month, sex, age)=(April 2020, male, 30s). However, with MLP, it is difficult to capture a wide range of trends (for example, periodicity of time series, etc.), and estimation accuracy may not be high.

Also, in the above N-dimensional convolution, inference is performed using an attribute value set and its adjacent attribute value sets. For example, attribute values such as {(year/month, gender, age) | year/month = March 2020, April 2020, May 2020, gender = male, female, age = 20s, 30s, 40s} Specific information (for example, the number of contracts) is estimated using a collection of sets as an input. However, N-dimensional convolution is difficult to apply to categorical attributes that do not have adjacency definition (eg, occupation, country name, etc.). Moreover, when the number of dimensions is N, the number of attribute value sets adjacent to a certain attribute value set is 3 ^N −1, and there is a problem that the number of adjacencies increases exponentially as the number of dimensions N increases.

An embodiment of the present invention has been made in view of the above points, and aims at estimating specific information from multidimensional data with high accuracy and low computational complexity.

In order to achieve the above object, a learning method according to one embodiment generates multidimensional data having an estimation target attribute indicating an attribute to be estimated and two or more non-estimation target attributes indicating attributes other than the estimation target attribute. a dimensionality reduction procedure for reducing the number of dimensions of the non-estimation target attribute of the multidimensional data; a binning procedure for binning the values of the non-estimation target attribute of the multidimensional data after the dimensionality reduction; An information addition procedure for adding predetermined additional information to multidimensional data after binning, and an inference model for estimating the value of the attribute to be estimated using the multidimensional data to which the additional information is added. A computer performs a learning procedure for learning the parameters.

It is possible to estimate specific information from multidimensional data with high accuracy and low computational complexity.

It is a figure which shows an example of the hardware constitutions of the inference apparatus which concerns on this embodiment. It is a figure showing an example of functional composition of an inference device concerning this embodiment. 6 is a flowchart showing an example of learning processing according to the embodiment; It is a figure which shows an example of the multi-dimensional data set with a correct answer in tabular form. FIG. 10 is a diagram showing an example of a multidimensional data set with a dimensionally reduced correct answer in tabular form; FIG. 4 is a diagram showing an example of a binned multidimensional data set with correct answers in tabular form. FIG. 4 is a diagram showing an example of a multidimensional data set for learning in tabular form; It is a figure which shows an example of an inference model. FIG. 10 is a diagram showing another example of an inference model; 6 is a flowchart showing an example of inference processing according to the embodiment; It is a figure which shows an example of a multi-dimensional data set without a correct answer in tabular form. FIG. 10 is a diagram showing an example of a dimension-reduced no-correct multidimensional data set in tabular form; FIG. 4 is a diagram showing an example of a binned non-correct multidimensional data set in tabular form. FIG. 4 is a diagram showing an example of a multidimensional data set for inference in tabular form;

An embodiment of the present invention will be described below. An inference device 10 capable of estimating specific information from multidimensional data with high accuracy and low computational complexity will be described.

<Hardware configuration>
First, the hardware configuration of the inference device 10 according to this embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of the hardware configuration of an inference device 10 according to this embodiment.

As shown in FIG. 1, the inference device 10 according to the present embodiment is realized by the hardware configuration of a general computer or computer system, and includes an input device 101, a display device 102, an external I/F 103, and a communication I/F. F 104 , processor 105 and memory device 106 . Each of these pieces of hardware is communicably connected via a bus 107 .

The input device 101 is, for example, a keyboard, mouse, touch panel, or the like. The display device 102 is, for example, a display. Note that the inference device 10 may not have at least one of the input device 101 and the display device 102, for example.

The external I/F 103 is an interface with an external device such as the recording medium 103a. The inference device 10 can perform reading, writing, etc. of the recording medium 103 a via the external I/F 103 . Examples of the recording medium 103a include CD (Compact Disc), DVD (Digital Versatile Disk), SD memory card (Secure Digital memory card), USB (Universal Serial Bus) memory card, and the like.

The communication I/F 104 is an interface for connecting the inference device 10 to a communication network. The processor 105 is, for example, various arithmetic units such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The memory device 106 is, for example, various storage devices such as HDD (Hard Disk Drive), SSD (Solid State Drive), RAM (Random Access Memory), ROM (Read Only Memory), and flash memory.

The inference device 10 according to the present embodiment has the hardware configuration shown in FIG. 1, so that learning processing and inference processing, which will be described later, can be realized. Note that the hardware configuration shown in FIG. 1 is merely an example, and the inference device 10 may have, for example, multiple processors 105 or multiple memory devices 106 .

<Functional configuration>
Next, the functional configuration of the inference device 10 according to this embodiment will be described with reference to FIG. FIG. 2 is a diagram showing an example of the functional configuration of the inference device 10 according to this embodiment.

As shown in FIG. 2, the inference device 10 according to this embodiment has a learning unit 201 and an inference unit 202 . These units are implemented by, for example, processing that one or more programs installed in the inference apparatus 10 cause the processor 105 to execute.

Further, the reasoning apparatus 10 according to the present embodiment includes a multidimensional data with correct answer storage unit 203, a learning dimensionality reduction model storage unit 204, a learning inference model storage unit 205, and a trained dimensionality reduction model storage unit 206. , a learned inference model storage unit 207 , a no-correct multidimensional data storage unit 208 , and an estimation result storage unit 209 . Each of these units is realized by the memory device 106, for example. At least one of these units may be realized by a storage device or the like connected to the inference device 10 via a communication network.

The learning unit 201 learns a dimensionality reduction model for reducing the number of dimensions of multidimensional data, and an inference model for estimating specific information from the multidimensional data subjected to dimensionality reduction by the dimensionality reduction model. .

The inference unit 202 uses the trained dimensionality reduction model and the trained inference model to estimate specific information from the multidimensional data.

The multidimensional data with correct answers storage unit 203 stores a set of multidimensional data with correct answers (hereinafter also referred to as a multidimensional data set with correct answers) used when learning the dimensionality reduction model and the inference model. A multidimensional data set with correct answers is a set of multidimensional data to which correct answers (that is, so-called teacher data) of specific information to be estimated by an inference model are added. Hereinafter, among the attributes of multidimensional data, an attribute that can take specific information to be estimated as a value is also referred to as an "estimation target attribute", and the other attributes are also referred to as "non-estimation target attributes". A specific example of the multidimensional data set with correct answers will be described later.

The learning dimensionality reduction model storage unit 204 stores a dimensionality reduction model to be learned by the learning unit 201 (hereinafter also referred to as a learning dimensionality reduction model). A dimensionality reduction model is a model for reducing the number of dimensions of multidimensional data, such as principal component analysis (PCA). In this embodiment, the dimensionality reduction model is PCA, but this is an example and other dimensionality reduction models may be used.

The learning inference model storage unit 205 stores an inference model to be learned by the learning unit 201 (hereinafter also referred to as a learning inference model). An inference model is a model for estimating an attribute value to be estimated from multidimensional data that has undergone dimensionality reduction or the like, and is, for example, a neural network. A specific example of the inference model will be described later.

The learned dimensionality reduction model storage unit 206 stores the learned dimensionality reduction model learned by the learning unit 201 .

The learned inference model storage unit 207 stores the learned inference model learned by the learning unit 201.

The non-correct answer multidimensional data storage unit 208 stores a set of non-correct answer multidimensional data to which estimation target attribute values are not assigned and multidimensional data with correct answer (hereinafter also referred to as a non-correct answer multidimensional data set). A multidimensional data set without correct answer is a multidimensional data set that includes multidimensional data to which an estimation target attribute value is not assigned. A specific example of the non-correct answer multidimensional data set will be described later.

The estimation result storage unit 209 stores the estimation result by the inference unit 202 (that is, the estimation target attribute value estimated from the non-correct multidimensional data).

Here, the learning unit 201 includes a dimension reduction unit 211, a binning unit 212, an information addition unit 213, and a learning processing unit 214.

The dimension reduction unit 211 receives the multidimensional data set with correct answer and the dimensionality reduction model for learning as input, and reduces the number of dimensions of non-estimation target attributes of each multidimensional data included in the multidimensional data set with correct answer.

Then, the dimensionality reduction unit 211 outputs a set of multidimensional data after this dimensionality reduction (hereinafter also referred to as a dimensionality-reduced multidimensional data set with correct answers) and a trained dimensionality reduction model. A specific example of the dimension-reduced correct multidimensional data set will be described later.

The binning unit 212 receives the dimension-reduced correct answer multidimensional data set as an input, and performs binning on non-estimation target attributes of each multidimensional data included in the dimension-reduced correct answer multidimensional data set. Then, the binning unit 212 outputs a set of multidimensional data after this binning (hereinafter also referred to as a binned multidimensional data set with correct answer). Binning is also called discretization, and divides the possible range of non-estimation target attributes contained in multidimensional data into bins of a certain interval, and determines which bin the value of the non-estimation target attribute belongs to. It is a method of replacing with a value that expresses

The information addition unit 213 receives the binned multidimensional data set with correct answer as input, and creates a learning multidimensional data set by adding additional information to each multidimensional data included in the binned multidimensional data set with correct answer. do. Here, the additional information refers to each non-estimation target attribute of each multidimensional data included in the binned correct answer multidimensional data set, while fixing the value of the non-estimation target attribute other than the non-estimation target attribute. It is information configured by the value of the inference target attribute when the value of the non-inference target attribute is changed.

Then, the information addition unit 213 outputs the learning multidimensional data set. A specific example of the learning multidimensional data set will be described later.

The learning processing unit 214 receives the multidimensional data set for learning and the inference model for learning as input, learns the inference model for learning, and creates a trained inference model. Then, the learning processing unit 214 outputs a learned inference model.

The inference unit 202 also includes a dimension reduction unit 221 , a binning unit 222 , an information addition unit 223 , and an estimation processing unit 224 .

The dimension reduction unit 221 receives the non-correct multidimensional data set and the trained dimensionality reduction model as input, and reduces the dimension of the non-estimation target attribute of each multidimensional data included in the non-correct multidimensional data set. Then, the dimension reduction unit 221 outputs a set of multidimensional data after the dimension reduction (hereinafter also referred to as a dimension-reduced non-correct multidimensional data set).

The binning unit 222 receives the dimension-reduced non-correct multidimensional data set as input, and performs binning on non-estimation target attributes of each multidimensional data included in the dimension-reduced non-correct multidimensional data set. Then, the binning unit 222 outputs a set of multidimensional data after this binning (hereinafter also referred to as a binned non-correct multidimensional data set).

The information addition unit 223 receives the binned non-correct answer multidimensional data set as an input, and creates an inference multidimensional data set by adding additional information to each multidimensional data included in the binned non-correct answer multidimensional data set. do.

The estimation processing unit 224 receives the inference multidimensional data set and the learned inference model as input, and estimates an inference target attribute value using the learned inference model. Then, the estimation processing unit 224 outputs the estimation target attribute value as an estimation result.

Note that each unit shown in FIG. 1 may be distributed among a plurality of devices. In particular, for example, the learning unit 201 and the inference unit 202 may be included in different devices. At this time, the device having the learning unit 201 may be called a “learning device”.

<Learning processing>
Next, the learning process according to this embodiment will be described with reference to FIG. FIG. 3 is a flowchart showing an example of learning processing according to this embodiment.

Step S101: First, the dimension reduction unit 211 of the learning unit 201 inputs the multidimensional data set with correct answer stored in the multidimensional data with correct answer storage unit 203 .

Here, a specific example of a multidimensional dataset with correct answers will be described with reference to FIG. FIG. 4 is a diagram showing an example of a multidimensional data set with correct answers in tabular form.

The example shown in FIG. 4 is a tabular representation of a set of multidimensional data having "year/month", "gender", "age", etc. as non-estimated attributes and "number of contracts" as an estimated target attribute. .

"Year and month" can take values indicating the year and month, such as "2019/4" and "2019/5". "Gender" can take the value of either "male" or "female". "Age" can take values indicating ages such as "teens", "twenties", and "thirties". The "number of contracts" takes as a value the number of contracts in the relevant year/month, gender, and age group.

For example, in the multidimensional data in the first row of the multidimensional data set with correct answer shown in FIG. , 200). Similarly, in the multidimensional data in the second row, (year/month, sex, age, ..., number of contracts) = (2019/5, female, twenties, ..., 100).

The multidimensional data set with correct answer shown in FIG. 4 is an example, and the present embodiment can be applied to multidimensional data sets having various attributes. Also, the values that each attribute can take can be defined in various ways. For example, in the example shown in FIG. 4, "year and month" can take values for each month of a certain year, but it may take values for every other month of a certain year. It may take a value. Similarly, the "age" is not limited to taking values separated by 10 years, for example, it may take values such as "young age group" or "old age group", or "20 years old to 25 years old". The value may be a range such as "years old" or a specific age value such as "20 years old". Also, the "number of contracts" may be the number of contracted customers or the total number of contracts.

In the following, each multidimensional data contained in the multidimensional data set with correct answers has various non-estimable attributes other than “year/month”, “gender”, and “age”, and there are a total of N non-estimable attributes. attribute. That is, each multidimensional data included in the multidimensional data set with correct answers has N non-estimation target attributes and one estimation target attribute.

Step S102: Next, the dimensionality reduction unit 211 of the learning unit 201 inputs the multidimensional data set with correct answer and the dimensionality reduction model for learning stored in the dimensionality reduction model for learning storage unit 204, Reduce the number of dimensions of non-estimable attributes of each multidimensional data contained in the multidimensional dataset. As a result, the dimensionality reduction model for learning is learned, and a learned dimensionality reduction model is obtained. In the following, as an example, it is assumed that the number of dimensions (N dimensions) of non-estimation target attributes has been reduced to three dimensions of the first principal component, the second principal component, and the third principal component by principal component analysis. Since the principal component analysis is a well-known method, detailed description thereof will be omitted.

Then, the dimension reduction unit 211 of the learning unit 201 outputs the dimension-reduced multidimensional data set with correct answer to the binning unit 212 and outputs the trained dimension reduction model to the trained dimension reduction model storage unit 206 . Note that the trained dimensionality reduction model, for example, converts N-dimensional data having N non-estimation target attributes into three-dimensional data having a first principal component, a second principal component, and a third principal component as attributes. It is the information representing the mapping and its parameters.

Here, a specific example of the dimension-reduced correct answer multidimensional data set will be described with reference to FIG. FIG. 5 is a diagram showing an example of a multidimensional data set with correct answers after dimensionality reduction in tabular form.

The example shown in FIG. 5 is a tabular representation of a set of multidimensional data in which non-estimable attributes of each multidimensional data included in the multidimensional data set with correct answer shown in FIG. 4 are reduced to three dimensions. . Each multidimensional data included in the dimensionally reduced multidimensional data set with correct answer shown in FIG. number” as an inference target attribute.

"First principal component", "second principal component" and "third principal component" take the values of the first principal component, second principal component and third principal component, respectively, in principal component analysis. These first to third principal components may be non-estimable attributes of each multidimensional data contained in the multidimensional data set with correct answer shown in FIG. 4, or newly defined attributes may be

For example, the multidimensional data in the first row of the multidimensional data set with the correct answer after the dimension reduction shown in FIG. 5 is the multidimensional data in the first row of the multidimensional data set with the correct answer shown in FIG. , (first principal component, second principal component, third principal component, number of contracts)=(53, 28, 103, 200). Similarly, the multidimensional data in the second row is obtained by reducing the dimensions of the multidimensional data in the second row of the multidimensional data set with correct answer shown in FIG. 3 principal components, number of contracts) = (24, 80, 9, 100).

It should be noted that the multidimensional data set with correct answer after dimension reduction shown in FIG. It is possible to In general, the smaller the number of dimensions after dimensionality reduction, the more the amount of calculation can be reduced, but the estimation accuracy at the time of inference is reduced. Therefore, the number of dimensions after dimension reduction is appropriately determined in consideration of the type of target task, the calculation time required for the task, and the like.

Step S103: Next, the binning unit 212 of the learning unit 201 receives the dimensionality-reduced correct multidimensional data set as input, and non-estimation target attributes of each multidimensional data included in the dimensionality-reduced correct multidimensional data set. binning. Then, the binning unit 212 of the learning unit 201 outputs the binned multidimensional data set with correct answer to the information adding unit 213 .

Here, a specific example of the binned multidimensional data set with correct answers will be described with reference to FIG. FIG. 6 is a diagram showing an example of a binned multidimensional data set with correct answers in tabular form.

The example shown in FIG. 6 is a tabular representation of a set of multidimensional data obtained by binning the non-estimable attributes of each multidimensional data contained in the multidimensional data set with the dimension-reduced correct answer shown in FIG. It is. Each multidimensional data included in the binned multidimensional data set with correct answer shown in FIG. ” as an inferred target attribute, and the values of each non-inferred target attribute are binned.

For example, the multidimensional data in the first row of the multidimensional data set with the binned correct answer shown in FIG. (first principal component, second principal component, third principal component, number of contracts)=(5, 3, 10, 200). Similarly, the multidimensional data in the second row is obtained by binning the values of the non-estimation target attribute of the multidimensional data in the second row of the multidimensional data set with correct answer after the dimension reduction shown in FIG. principal component, second principal component, third principal component, number of contracts)=(2, 8, 1, 100).

It should be noted that the binned multidimensional data set with correct answer shown in FIG. 6 is an example, and for example, the interval width of the bins during binning can be appropriately set to any value.

Step S104: Next, the information addition unit 213 of the learning unit 201 receives the binned multidimensional data set with correct answer as input, and adds additional information to each multidimensional data included in the binned multidimensional data set with correct answer. Create a given training multidimensional dataset. Then, the information addition unit 213 of the learning unit 201 outputs the learning multidimensional data set to the learning processing unit 214 .

Note that the additional information is, as described above, fixed values of non-estimation target attributes other than the non-estimation target attributes for each non-estimation target attribute of each multidimensional data included in the binned multidimensional data set with correct answers. It is information composed of the value of the inference target attribute when the value of the non-inference target attribute is changed (within the range of values that the non-inference target attribute can take) while the value of the inference target attribute is changed. The value of such an inference target attribute is the binned correct answer of the value of the inference target attribute when the value of the non-estimation target attribute is changed while the value of the non-estimation target attribute other than the non-estimation target attribute is fixed. It is obtained by searching from a multidimensional data set with and aggregating those estimated target attribute values.

Here, a specific example of the multidimensional data set for learning will be described with reference to FIG. FIG. 7 is a diagram showing an example of a learning multidimensional data set in tabular form.

The example shown in FIG. 7 is a tabular representation of a set of multidimensional data in which additional information is added to each piece of multidimensional data included in the binned multidimensional data set with correct answers shown in FIG. Each piece of multidimensional data included in the multidimensional data set for learning shown in FIG. additional information is added.

"Fixed other than 1st principal component" tabulates the value of the non-estimable attribute (number of contracts) when the value of the 1st principal component is changed while the values of the 2nd and 3rd principal components are fixed. It is what I did. In the example shown in FIG. 7, the number of contracts is tabulated when the value of the first principal component is changed to "0", "1", etc., respectively.

"Fixed other than 2nd principal component" tabulates the value of the non-estimable attribute (number of contracts) when the value of the 2nd principal component is changed while the values of the 1st and 3rd principal components are fixed. It is what I did. In the example shown in FIG. 7, the number of contracts is tabulated when the value of the second principal component is changed to "0", "1", etc., respectively.

"Fixed other than the 3rd principal component" aggregates the value of the non-estimable attribute (number of contracts) when the value of the 3rd principal component is changed while the values of the 1st and 2nd principal components are fixed. It is what I did. In the example shown in FIG. 7, the number of contracts is totaled when the value of the third principal component is changed to "0", "1", etc., respectively.

For example, "0" in "Fixed other than the first principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. The number of contracts is aggregated when the value of the first principal component of the eye multidimensional data set is changed to "0". That is, "0" in the "fixed other than the first principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. 1 principal component, 2nd principal component, 3rd principal component)=(0, 3, 10).

Similarly, "1" of "fixed other than the first principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. The number of contracts is aggregated when the value of the first principal component of the multidimensional data set in the row is changed to "1". That is, "1" in the "fixed other than the first principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. 1 principal component, 2nd principal component, 3rd principal component)=(1, 3, 10).

Similarly, "0" of "fixed other than the second principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. The number of contracts is aggregated when the value of the second principal component of the multidimensional data set in the row is changed to "0". That is, "0" in the "fixed other than the second principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. 1st principal component, 2nd principal component, 3rd principal component)=(5, 0, 10).

Similarly, "1" of "fixed other than the second principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. The number of contracts is aggregated when the value of the second principal component of the multidimensional data set in the row is changed to "1". That is, "1" in the "fixed other than the second principal component" of the multidimensional data in the first row of the multidimensional data set for learning shown in FIG. 1 principal component, 2nd principal component, 3rd principal component)=(5, 1, 10).

The same applies to other attributes of additional information. In the following, a non-estimation target attribute value set is defined as a set of values of the "first principal component", "second principal component", and "third principal component" of each multidimensional data included in the multidimensional data set for learning. It is also called W, and the set of additional information is represented by X.

Note that the multidimensional data set for learning shown in FIG. 7 is an example. Multidimensional data may be created with unknowns (for example, set to "-" or null value).

Step S105: Next, the learning processing unit 214 of the learning unit 201 inputs the learning multidimensional data set and the learning inference model stored in the learning inference model storage unit 205, and and create a trained inference model. That is, the learning processing unit 214 of the learning unit 201 receives a vector wεW representing a set of non-estimation target attribute values and additional information xεX corresponding to w, and correctly estimates the estimation target attribute value y. (that is, to minimize the error between the estimated target attribute value y estimated by the learning inference model and its correct answer), the parameters of the learning inference model are set using a known error backpropagation method, etc. learn. Note that x corresponding to w is x in the same row as the set of non-estimation target attribute values represented by w in the learning multidimensional data set.

Here, the inference model is not particularly limited, but for example, a neural network as shown in Fig. 8 may be used as the inference model. The neural network shown in FIG. 8 is a model that receives as input a vector w representing a set of non-estimation target attribute values and its additional information x, and outputs an estimation target attribute value y.

Also, for example, a neural network as shown in FIG. 9 may be used as an inference model. The neural network shown in FIG. 9 receives a vector w representing a set of non-estimation target attribute values and its additional information x, and outputs a vector w′ representing a set of estimation target attribute values y and non-estimation target attribute values. It is a model that When using this model, in addition to the attribute value y to be estimated, the parameters of the inference model for learning are learned so that w' correctly reproduces w. That is, in addition to the attribute value y to be estimated, the parameters of the inference model for learning are learned so that w' that reproduces the original w can be estimated by multitasking. By using a neural network as shown in FIG. 9 as an inference model, it is possible to appropriately extract features from the additional information x. High estimation accuracy can be expected.

Step S106: Finally, the learning processing unit 214 of the learning unit 201 outputs the learned inference model to the learned inference model storage unit 207.

<Inference processing>
Next, inference processing according to this embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing an example of inference processing according to this embodiment.

Step S201: First, the dimension reduction unit 221 of the inference unit 202 inputs the multidimensional data without correct answers stored in the multidimensional data storage unit 208 without correct answers.

Here, a specific example of a non-correct multidimensional data set will be described with reference to FIG. FIG. 11 is a diagram showing an example of a non-correct multidimensional data set in tabular form.

The example shown in FIG. 11 is a tabular representation of a set of multidimensional data having "year/month", "sex", "age", etc. as non-estimation target attributes and "number of contracts" as an estimation target attribute. . In the multidimensional data in the first row of the multidimensional data set without correct answer shown in FIG. 200). On the other hand, in the multidimensional data in the second row, (year/month, sex, age, ..., number of contracts) = (2020/5, female, twenties, ..., -). That is, in the multidimensional data in the second row, the number of contracts, which is the attribute value to be estimated, is unknown. In this way, the non-correct multidimensional data set includes at least multidimensional data whose attribute values to be estimated are unknown.

Step S202: Next, the dimensionality reduction unit 221 of the inference unit 202 inputs the non-correct multidimensional data set and the learned dimensionality reduction model stored in the learned dimensionality reduction model storage unit 206. The dimensionality reduction model is used to reduce the number of dimensions of non-estimation target attributes of each multidimensional data included in the non-correct answer multidimensional data set. Then, the dimension reduction unit 221 of the inference unit 202 outputs the dimension-reduced non-correct multidimensional data set to the binning unit 212 .

Here, a specific example of the dimension-reduced non-correct multidimensional data set will be described with reference to FIG. FIG. 12 is a diagram showing an example of a dimension-reduced non-correct multidimensional data set in tabular form.

The example shown in FIG. 12 is a tabular representation of a set of multidimensional data obtained by reducing the non-estimable attributes of each multidimensional data contained in the non-correct multidimensional data set shown in FIG. 11 to three dimensions. . Each multidimensional data included in the dimension-reduced non-correct multidimensional data set shown in FIG. number” as an inference target attribute.

For example, the multidimensional data in the first row of the dimension-reduced non-correct multidimensional data set shown in FIG. 12 is the multidimensional data in the first row of the non-correct multidimensional data set shown in FIG. , (first principal component, second principal component, third principal component, number of contracts)=(58, 21, 109, 200). Similarly, the multidimensional data in the second row is obtained by reducing the dimensions of the multidimensional data in the second row of the non-correct answer multidimensional data set shown in FIG. 3 principal components, number of contracts) = (20, 81, 6, -).

Step S203: Next, the binning unit 222 of the inference unit 202 receives the dimension-reduced non-correct multidimensional data set as input, and the non-estimation target attribute of each multidimensional data included in the dimension-reduced non-correct multidimensional data set. binning. Then, the binning unit 222 of the inference unit 202 outputs the binned non-correct multidimensional data set to the information adding unit 223 .

Here, a specific example of the binned non-correct multidimensional data set will be described with reference to FIG. FIG. 13 is a diagram showing an example of a binned non-correct multidimensional data set in tabular form.

The example shown in FIG. 13 is a tabular representation of a set of multidimensional data obtained by binning non-estimable attributes of each multidimensional data included in the dimension-reduced non-correct multidimensional data set shown in FIG. It is. Each multidimensional data included in the binned non-correct answer multidimensional data set shown in FIG. ” as an inferred target attribute, and the values of each non-inferred target attribute are binned.

For example, the multidimensional data in the first row of the dimension-reduced no-correct multidimensional data set shown in FIG. Attribute values are binned, and (first principal component, second principal component, third principal component, number of contracts)=(5, 3, 10, 200). Similarly, the multidimensional data in the second row is obtained by binning the values of the non-estimation target attribute of the multidimensional data in the second row of the dimension-reduced non-correct multidimensional data set shown in FIG. principal component, second principal component, third principal component, number of contracts)=(2, 8, 1, -).

Step S204: Next, the information addition unit 223 of the inference unit 202 receives the binned non-correct answer multidimensional data set as input, and adds additional information to each multidimensional data included in the binned non-correct answer multidimensional data set. Create a given inference multidimensional dataset. Then, the information addition unit 223 of the inference unit 202 outputs the inference multidimensional data set to the estimation processing unit 224 .

Here, a specific example of the multidimensional data set for inference will be described with reference to FIG. FIG. 14 is a diagram showing an example of a multidimensional data set for inference in tabular form.

The example shown in FIG. 14 is a tabular representation of a set of multidimensional data in which additional information is added to each piece of multidimensional data included in the binned non-correct answer multidimensional data set shown in FIG. Each multidimensional data included in the multidimensional data set for inference shown in FIG. additional information is added.

For example, "0" in "Fixed other than the first principal component" of the multidimensional data in the first row of the multidimensional data set for inference shown in FIG. The number of contracts is aggregated when the value of the first principal component of the eye multidimensional data set is changed to "0". That is, "0" in the "fixed other than the first principal component" of the multidimensional data in the first row of the multidimensional data set for inference shown in FIG. 1 principal component, 2nd principal component, 3rd principal component)=(0, 3, 10).

Similarly, "0" in "Fixed other than the first principal component" of the multidimensional data in the second row of the multidimensional data set for inference shown in FIG. The number of contracts is aggregated when the value of the first principal component of the multidimensional data set in the row is changed to "0". That is, "0" in the "fixed other than the first principal component" of the multidimensional data in the second row of the multidimensional data set for inference shown in FIG. 1 principal component, 2nd principal component, 3rd principal component)=(0, 8, 1).

Here, when the value of the non-estimation target attribute other than a certain non-estimation target attribute is fixed and the value of the non-estimation target attribute is changed to a certain value, if the value of the inference target attribute does not exist, , "-" or a null value is set in the additional information. For example, "-" is set to "1" of "fixed except for the first principal component" of the multidimensional data in the first row of the inference multidimensional data set shown in FIG. This is because the number of contracts when (first principal component, second principal component, third principal component) = (1, 3, 10) does not exist in the binned non-correct multidimensional data set shown in FIG. It's for.

The same applies to other attributes of additional information. In the following, as in the case of learning, a set of sets of values of “first principal component”, “second principal component” and “third principal component” of each multidimensional data included in the inference multidimensional data set is It is also called non-estimation target attribute value set W, and the set of additional information is represented by X.

Step S205: Next, the estimation processing unit 224 of the inference unit 202 inputs the inference multidimensional data set and the learned inference model stored in the learned inference model storage unit 207, and Estimates the attribute value to be estimated by That is, the estimation processing unit 224 of the inference unit 202 receives as input a vector wεW representing a set of non-estimation target attribute values and additional information xεX corresponding to w, and uses the learned inference model to determine the estimation target attribute. Estimate the value y.

Step S<b>206 : Finally, the estimation processing unit 224 of the inference unit 202 outputs the estimation result of the learned inference model (that is, the estimation target attribute value y) to the estimation result storage unit 209 .

<Summary>
As described above, the inference apparatus 10 according to the present embodiment changes the value of each non-estimation target attribute while fixing the value of the non-estimation target attribute other than the non-estimation target attribute. After creating additional information composed of the value of the attribute to be estimated at the time, learning and inference are performed using this additional information as well. As a result, global features can be extracted, enabling more accurate inference.

In addition, the inference device 10 according to the present embodiment reduces the number of dimensions of non-estimation target attributes and reduces the number of attribute values of each non-estimation target attribute by binning before creating additional information. This makes it possible to reduce the amount of calculation even when the number of non-estimation target attributes and the number of attribute values are large. For example, if the number of non-estimable attributes before dimensionality reduction is N and the average of the attribute values is M, calculation must be performed for N×M attribute values without dimensionality reduction and binning. . On the other hand, if the number of non-estimable attributes after dimensionality reduction is n<N and the average of attribute values after binning is m<M, in this embodiment, if calculation is performed for n×m attribute values, It is often possible to learn and reason with less computational effort.

The present invention is not limited to the specifically disclosed embodiments described above, and various modifications, alterations, combinations with known techniques, etc. are possible without departing from the scope of the claims. .

10 inference device 101 input device 102 display device 103 external I/F
103a recording medium 104 communication I/F
105 processor 106 memory device 107 bus 201 learning unit 202 inference unit 203 multidimensional data storage unit with correct answer 204 dimensionality reduction model storage unit for learning 205 inference model storage unit for learning 206 learned dimensionality reduction model storage unit 207 learned inference model storage unit Unit 208 Multidimensional data storage unit without correct answer 209 Estimation result storage unit 211 Dimension reduction unit 212 Binning unit 213 Information addition unit 214 Learning processing unit 221 Dimension reduction unit 222 Binning unit 223 Information addition unit 224 Estimation processing unit

Claims

an input step of inputting multidimensional data having an inference target attribute indicating an inference target attribute and two or more non-estimation target attributes indicating attributes other than the inference target attribute;
a dimension reduction procedure for reducing the number of dimensions of non-estimation target attributes of the multidimensional data;
a binning procedure for binning the values of the non-estimable attribute of the multidimensional data after the dimensionality reduction;
an information addition procedure for adding predetermined additional information to the binning multidimensional data;
a learning procedure for learning parameters of an inference model for estimating the value of the attribute to be estimated using the multidimensional data to which the additional information is added;
a computer-implemented learning method.
The information addition procedure includes:
For each of the non-estimation target attributes of the multidimensional data after binning, a total value of the values of the estimation target attributes when the values of the non-estimation target attributes other than the non-estimation target attributes are changed is added as the additional information. The learning method according to claim 1, wherein
The learning procedure includes:
Error between the value of the estimated attribute estimated by the inference model using the value of the non-estimated attribute as an input and the correct value of the estimated attribute, and the non-estimated attribute reproduced by the inference model. 3. The learning method according to claim 1 or 2, wherein parameters of said inference model are learned so as to minimize an error between the value of and the value of said non-estimated attribute input to said inference model.
an input step of inputting multidimensional data having an inference target attribute indicating an inference target attribute and two or more non-estimation target attributes indicating attributes other than the inference target attribute;
a dimension reduction procedure for reducing the number of dimensions of non-estimation target attributes of the multidimensional data;
a binning procedure for binning the values of the non-estimable attribute of the multidimensional data after the dimensionality reduction;
an information addition procedure for adding predetermined additional information to the binning multidimensional data;
an estimation step of estimating the value of the attribute to be estimated by a pre-trained inference model using the multidimensional data to which the additional information is added;
is a computer-implemented inference method.
an input unit for inputting multidimensional data having an inference target attribute indicating an inference target attribute and two or more non-estimation target attributes indicating attributes other than the inference target attribute;
a dimension reduction unit that reduces the number of dimensions of non-estimation target attributes of the multidimensional data;
a binning unit for binning the values of the non-estimation target attribute of the multidimensional data after the dimensionality reduction;
an information addition unit that adds predetermined additional information to the binning multidimensional data;
a learning unit that learns parameters of an inference model for estimating the value of the attribute to be estimated using the multidimensional data to which the additional information is added;
A learning device having
an input unit for inputting multidimensional data having an inference target attribute indicating an inference target attribute and two or more non-estimation target attributes indicating attributes other than the inference target attribute;
a dimension reduction unit that reduces the number of dimensions of non-estimation target attributes of the multidimensional data;
a binning unit for binning the values of the non-estimation target attribute of the multidimensional data after the dimensionality reduction;
an information addition unit that adds predetermined additional information to the multidimensional data after binning;
an estimating unit that estimates the value of the attribute to be estimated by a pre-trained inference model using the multidimensional data to which the additional information is added;
A reasoning device with
A program that causes a computer to execute the learning method according to any one of claims 1 to 3 or the inference method according to claim 4.