JP2017182319A - Machine learning device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a machine learning device that can generate a neural network of an appropriate scale.SOLUTION: An intermediate layer selecting unit 12 selects intermediate layers 22 and 23 out of intermediate layers 22 to 25 contained in an already learned neural network 3A. A tentative output layer adding unit 13 generates a neural network 3B by adding a tentative output layer 32 connected to an intermediate layer 22 and a tentative output layer 33 connected to an intermediate layer 23 to the neural network 3A. An arithmetic unit 15 performs arithmetic operation using the neural network 3B and acquires a tentative output value outputted from each of the tentative output layers 32 and 33. A deletion determining unit 16 calculates levels of similarity by using the tentative output value acquired by the arithmetic unit 15, and determines which intermediate layer is to be deleted out of the intermediate layers 22 and 23. A reconfiguration unit 17 deletes one or the other of the intermediate layers from the neural network 3A.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a machine learning device that performs learning of a neural network.

  The neural network is employed in an object detection device that detects a predetermined object such as a person from an image taken by a camera, an analysis device that analyzes data measured by a sensor, and the like. For example, when a neural network is used for an object detection device, a machine learning device causes a neural network to learn features of a predetermined object included in an image. An object detection apparatus is created by mounting a neural network that has finished learning on a computer or the like.

  A neural network tends to have higher learning accuracy as its scale increases. However, as the scale of the neural network increases, the amount of computation of the neural network increases. In order to apply the neural network to an object detection device or a data analysis device, it is desirable that the amount of calculation of the neural network is as small as possible. Therefore, it is desired to develop a technique for creating a neural network with high accuracy and a small amount of calculation.

  Non-Patent Document 1 discloses a technique for learning a small-scale neural network using an output result of a learned large-scale neural network. The error back-propagation method (back propagation) is used for learning of a small-scale neural network. The error back propagation method is a supervised learning algorithm.

“Distilling the Knowledge in a Neural Network”, [online], [searched February 9, 2016], Internet <URL: https://www.cs.toronto.edu/~hinton/absps/distillation.pdf>

  When the neural network is used for an object detection device, a data analysis device, or the like, it is necessary to determine the size of the neural network in advance. Specifically, the scale of the neural network is determined by the number of layers that the neural network has and the number of nodes that each layer has. As described above, the learning accuracy of the neural network can be improved by increasing the scale of the neural network. Therefore, the scale of the neural network is determined based on learning accuracy required in advance.

  However, in order to achieve the required learning accuracy, there is a case where there is a margin in the scale of the neural network. That is, the scale of the neural network is set excessively compared to the required learning accuracy. If the scale of the neural network is excessive, the learned neural network may be performing the same calculation repeatedly. In this case, it is desirable to reduce the amount of calculation by reducing the learned neural network to an appropriate scale according to the required learning accuracy. However, no technology has been developed to reduce the scale of a learned neural network.

  An object of the present invention is to provide a machine learning device capable of creating a neural network having an appropriate scale.

  In order to solve the above-mentioned problem, an invention according to claim 1 is a machine learning device, comprising: an acquisition unit that acquires a learned neural network including at least two intermediate layers; and an intermediate between the at least two intermediate layers. From the intermediate layer selector that selects the first intermediate layer and the second intermediate layer, the first temporary output layer having a temporary output node connected to the node of the first intermediate layer, and the second intermediate layer A temporary output layer adding unit that generates a temporary network by adding a second temporary output layer having a temporary output node connected to the node having the temporary output node; An operation using a network is executed, and the first temporary output value output from the temporary output node included in the first temporary output layer and the temporary output node included in the second temporary output layer are output. The degree of similarity of the function of the first intermediate layer to the function of the second intermediate layer is shown using the arithmetic unit for obtaining the second temporary output value, and the first temporary output value and the second temporary output value. Calculating a similarity, and determining whether to delete the one intermediate layer based on the calculated similarity; and when the deletion determining unit determines to delete the first intermediate layer, the neural network A reconfiguration unit that deletes the first intermediate layer from the network.

  A second aspect of the present invention is the machine learning device according to the first aspect, wherein the first intermediate layer is disposed closer to the input layer included in the neural network than the second intermediate layer.

  The invention according to claim 3 is the machine learning device according to claim 1 or 2, wherein the number of temporary output nodes included in the first temporary output layer is the temporary output included in the second temporary output layer. Same as node.

  The invention according to claim 4 is the machine learning device according to claim 3, wherein the number of temporary output nodes of the first temporary output layer is the same as the number of output nodes of the output layer in the neural network. It is.

  A fifth aspect of the present invention is the machine learning device according to any one of the first to fourth aspects, wherein the provisional output layer adding unit outputs nodes in the second intermediate layer in the neural network. If the layer is connected to a node, the output layer is determined to be used as the second temporary output layer.

  A sixth aspect of the present invention is the machine learning device according to any one of the first to fifth aspects, further comprising a node included in the first intermediate layer and a node included in the first temporary output layer. Learning to update the weighting factor set in the signal path between and the weighting factor set in the signal path between the node of the second intermediate layer and the node of the second temporary output layer An additional learning unit to be executed.

  A seventh aspect of the present invention is the machine learning device according to any one of the first to sixth aspects, wherein the reconfiguration unit counts the first intermediate layer from the input layer of the neural network. And k (k is a natural value of 2 to m−1) th layer, the node of the (k−1) th layer is newly connected to the node of the (k + 1) th layer, Learning for updating the weighting factor set in the signal path between the node of the (k-1) th layer and the node of the (k + 1) th layer is executed.

  The invention according to claim 8 is a machine learning method, wherein a learned neural network including at least two intermediate layers is obtained, and a first intermediate layer and a second intermediate layer are selected from the at least two intermediate layers. Selecting an intermediate layer; a first temporary output layer having a temporary output node connected to a node of the first intermediate layer; and a first output node having a temporary output node connected to a node of the second intermediate layer. 2 temporary output layers are added to the neural network to generate a temporary network; test data is input to the temporary network and an operation using the temporary network is performed; Obtaining a first temporary output value output from the temporary output node having and a second temporary output value output from the temporary output node of the second temporary output layer. Then, using the first temporary output value and the second temporary output value, a degree of similarity indicating the degree of similarity of the function of the first intermediate layer to the function of the second intermediate layer is calculated, and based on the calculated similarity Determining whether or not to delete the first intermediate layer, and deleting the first intermediate layer from the neural network when it is determined to delete the first intermediate layer.

  The machine learning device according to the present invention acquires a learned neural network. The intermediate layer selection unit selects two intermediate layers included in the acquired neural network. The temporary output layer adding unit adds a temporary output layer connected to each of the two selected intermediate layers to the neural network to generate a temporary network. A temporary output value is output from the temporary output layer by a calculation using the temporary network. The deletion determining unit calculates the similarity between the two intermediate layers based on the temporary output value, and determines whether one of the two intermediate layers can be deleted based on the calculated similarity. Judging. When it is determined that one of the intermediate layers can be deleted, the reconstruction unit deletes one of the intermediate layers from the neural network. As a result, the machine learning device can generate a neural network of an appropriate scale according to the required learning accuracy.

It is a functional block diagram which shows the structure of the machine learning apparatus which concerns on embodiment of this invention. It is a figure which shows an example of a structure of the neural network input into the machine learning apparatus shown in FIG. It is a flowchart which shows operation | movement of the machine learning apparatus shown in FIG. It is a figure which shows the temporary output layer connected to two intermediate | middle layers selected by the intermediate | middle layer selection part shown in FIG. It is a figure which shows the network which deleted the intermediate | middle layer from the neural network shown in FIG. It is a figure which shows the temporary output layer added to the neural network shown in FIG. It is a figure which shows the network which deleted the intermediate | middle layer from the neural network shown in FIG. It is a functional block diagram which shows the structure of the modification of the machine learning apparatus shown in FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

{1. Outline of Machine Learning Device 100}
FIG. 1 is a functional block diagram showing a configuration of a machine learning device 100 according to an embodiment of the present invention. As shown in FIG. 1, the machine learning device 100 inputs a learned neural network 3A. The machine learning device 100 specifies an intermediate layer that can be deleted from the intermediate layers of the neural network 3A. The machine learning device 100 outputs a neural network in which the identified intermediate layer is deleted from the neural network 3A.

  FIG. 2 is a diagram illustrating an example of the configuration of the neural network 3 </ b> A input to the machine learning device 100. As shown in FIG. 2, the neural network 3 </ b> A is a six-layer perceptron, and includes an input layer 21, intermediate layers 22 to 25, and an output layer 26. The input layer 21 includes input nodes 21a to 21d. The intermediate layer 22 includes nodes 22a to 22d. The intermediate layer 23 includes nodes 23a to 23d. The intermediate layer 24 includes nodes 24a to 24d. The intermediate layer 25 includes nodes 25a to 25d. The output layer 26 includes output nodes 26a to 26c. The output nodes 26a to 26c output normal output values 46a to 46c. Hereinafter, the normal output values 46a to 46c are collectively referred to as the normal output value 46.

  The neural network 3A has been learned as described above, and is used, for example, to detect a plurality of types of objects from an image taken by a camera. The plurality of types of objects are, for example, dogs, cats, and other objects. The learning algorithm of the neural network 3A is not particularly limited. Transfer learning may be used in the learning of the neural network 3A.

  When the neural network 3A shown in FIG. 2 is input, the machine learning device 100 selects two intermediate layers from the intermediate layers 22-25. The machine learning device 100 determines whether one of the two selected intermediate layers can be deleted based on the similarity between the two selected intermediate layers. A method for calculating the similarity will be described later.

  For example, the machine learning device 100 selects the intermediate layers 22 and 23 in the neural network 3A, and calculates the similarity between the intermediate layers 22 and 23. The machine learning device 100 determines to delete the intermediate layer 22 among the intermediate layers 22 and 23 based on the calculated similarity. The machine learning device 100 generates a network obtained by deleting the intermediate layer 22 from the neural network 3A as a neural network 3D. The neural network 3D has the same learning accuracy as the neural network 3A and is smaller than the scale of the neural network 3A. As described above, the machine learning device 100 can create a neural network 3D having an appropriate scale according to the required learning accuracy.

{2. Configuration of Machine Learning Device 100}
As illustrated in FIG. 1, the machine learning device 100 includes a network acquisition unit 11, an intermediate layer selection unit 12, a temporary output layer addition unit 13, an additional learning unit 14, a calculation unit 15, and a deletion determination unit 16. The reconfiguration unit 17 and the end determination unit 18 are provided.

  The network acquisition unit 11 acquires a learned neural network 3A. The network acquisition unit 11 may receive the neural network 3A from another computer via a LAN (Local Area Network), or may acquire it from a nonvolatile recording medium such as a Blu-ray disk. Alternatively, the network acquisition unit 11 may generate the learned neural network 3A itself.

  The intermediate layer selection unit 12 selects two intermediate layers from the intermediate layers 22 to 25 included in the neural network 3 </ b> A acquired by the network acquisition unit 11.

  The temporary output layer adding unit 13 adds a temporary output layer connected to each of the two selected intermediate layers to the neural network 3A. The temporary output layer adding unit 13 outputs a network obtained by adding a temporary output layer to the neural network 3A as a neural network 3B. For example, when the intermediate layer selecting unit 12 selects the intermediate layers 22 and 23, the temporary output layer adding unit 13 includes the temporary output layer having nodes connected to the respective nodes included in the intermediate layer 22, and the intermediate layer 23. A temporary output layer having a node connected to each node is added to the neural network 3A.

  The additional learning unit 14 uses the learning data 4 to perform additional learning of the neural network 3B. The learning data 4 is, for example, an image obtained by shooting any one of a dog, a cat, and other objects. The additional learning unit 14 updates the weighting coefficient of the signal path newly provided by adding the temporary output layer among the signal paths in the neural network 3B. An error back-propagation method (back propagation) is used for updating the weighting factor. The additional learning unit 14 outputs, as the neural network 3C, a network in which the signal path weight coefficient in the neural network 3B is updated.

  The calculation unit 15 inputs the test data 5 to the neural network 3C and executes a calculation using the neural network 3C. Similarly to the learning data 4, the test data 5 is an image in which any one of a dog, a cat, and other objects is photographed. The neural network 3C outputs a normal output value 46 from the output nodes 26a to 26c included in the output layer 26 as a calculation result. The neural network 3C outputs a temporary output value from one temporary output layer and outputs a temporary output value from the other temporary output layer.

  The deletion determination unit 16 calculates the similarity between the two intermediate layers selected by the intermediate layer selection unit 12 based on the temporary output value. A method for calculating the similarity will be described later. The deletion determination unit 16 determines whether or not one of the two intermediate layers selected by the intermediate layer selection unit 12 can be deleted based on the calculated similarity.

  When the reconfiguration unit 17 determines that the deletion determination unit 16 can delete any one of the two intermediate layers, the reconfiguration unit 17 deletes either one of the intermediate layers from the neural network 3A. A neural network 3D is generated. The reconfiguration unit 17 performs relearning of the generated neural network 3D. In the relearning of the neural network 3D, the reconfiguration unit 17 uses an algorithm used for learning of the neural network 3A.

  The end determination unit 18 determines whether to further delete the intermediate layer from the learned neural network 3D. Specifically, the end determination unit 18 determines the number of intermediate layers deleted from the neural network 3A by subtracting the number of intermediate layers included in the neural network 3D from the number of intermediate layers included in the neural network 3A. To do.

  When the number of deleted intermediate layers does not reach the preset end reference value, the end determination unit 18 determines that the intermediate layers can be further deleted from the neural network 3D. The end determination unit 18 outputs the neural network 3D to the intermediate layer selection unit 12 and the temporary output layer addition unit 13. Thereby, the machine learning device 100 repeats the process of determining whether or not the intermediate layer can be deleted from the neural network 3D.

  When the number of deleted intermediate layers reaches the end reference value, the end determination unit 18 determines that the intermediate layers cannot be further deleted from the neural network 3D. In this case, the end determination unit 18 outputs the neural network 3D to the outside of the machine learning device 100. The neural network 3D output to the outside of the machine learning device 100 is used for an object detection device (not shown) that identifies dogs, cats, and other objects.

{3. Operation of Machine Learning Device 100}
Hereinafter, the operation of the machine learning device 100 will be described in detail by taking as an example the case where the neural network 3A shown in FIG. 2 is input to the machine learning device 100. FIG. 3 is a flowchart showing the operation of the machine learning device 100.

{3.1. Delete intermediate layer 22}
First, the operation of the machine learning device 100 will be described by taking as an example the case where the intermediate layer 22 is deleted from the neural network 3A shown in FIG.

{3.1.1. Acquisition of neural network 3A}
As shown in FIG. 3, the network acquisition unit 11 acquires the neural network 3A input to the machine learning device 100 (step S1).

  The neural network 3A is data including data defining nodes included in each layer and a weighting factor of a signal path connecting the two nodes. When two layers are adjacent to each other, the signal path connects one node included in one layer and a node included in the other layer. Since the neural network 3A has been learned, the weight counts included in the neural network 3A have already been adjusted to identify dogs, cats and other objects.

  The neural network 3A shown in FIG. 2 is a six-layer perceptron, but the configuration of the neural network 3A is not limited to this. The neural network 3A only needs to include at least two intermediate layers. That is, the number of layers provided in the neural network 3A may be four or more. The number of nodes included in each layer of the neural network 3A is not particularly limited.

  In the neural network 3 </ b> A shown in FIG. 2, a “section” of a signal path in which a weight count is set is defined. Here, “section” is a convenient name for indicating the position of the signal path in the neural network 3A. Specifically, the section A <b> 1 is a section from the input layer 21 to the intermediate layer 22. The section B1 is a section from the intermediate layer 22 to the intermediate layer 23. The section C <b> 1 is a section from the intermediate layer 23 to the intermediate layer 24. The section D1 is a section from the intermediate layer 24 to the intermediate layer 25. The section E1 is a section from the intermediate layer 25 to the output layer 26. Next, directions in the neural network 3A are defined. Focusing on an arbitrary intermediate layer included in the neural network 3A, with reference to the target intermediate layer, the direction in which the input layer 21 is located is defined as "downward", and the direction in which the output layer 26 is located is defined as "upward Is defined.

{3.1.2. Middle layer selection}
The intermediate layer selection unit 12 selects two intermediate layers from the intermediate layers 22 to 25 included in the neural network 3A (step S2). For example, the intermediate layer selection unit 12 selects the intermediate layers 22 and 23. The intermediate layer selection unit 12 preferably selects two intermediate layers adjacent to each other. The reason will be described below. As described above, when the machine learning device 100 can identify two intermediate layers having similar functions among the intermediate layers 22 to 25 included in the neural network 3A, any one of the two intermediate layers may be specified. One of them is deleted from the neural network 3A.

  In the neural network 3A, the closer the distance between the two intermediate layers is, the closer the functions of the two intermediate layers are. The functions of each intermediate layer also change depending on the use of the neural network 3A. However, the intermediate layer close to the input layer 21 has a function of classifying abstract concepts, and the intermediate layer close to the output layer 26 is specific. It can be understood that it has the function of classifying concepts.

  For example, consider a case where the neural network 3A has a function of detecting any one of a dog, a cat, and other objects from a photographed image. In this case, the intermediate layer close to the input layer 21 determines whether the object in the image is an animal, and the intermediate layer close to the output layer 26 determines whether the animal in the image is a dog or a cat. It can be interpreted as discriminating.

  Accordingly, in the neural network 3A, two adjacent intermediate layers are expected to have functions similar to each other. For this reason, it is desirable for the intermediate layer selection unit 12 to select two adjacent intermediate layers. However, the two selected intermediate layers may not be adjacent to each other.

{3.1.3. Add temporary output layer}
The intermediate layer selection unit 12 outputs intermediate layer selection data for specifying the two intermediate layers selected in step S <b> 2 to the temporary output layer addition unit 13. The temporary output layer adding unit 13 adds two temporary output layers to the neural network 3A based on the intermediate layer selection data (step S3).

  FIG. 4 is a diagram showing a neural network 3B having a temporary output layer. A neural network 3B shown in FIG. 4 is generated by adding temporary output layers 32 and 33 to the neural network 3A shown in FIG. In FIG. 4, the intermediate layers 24 and 25 and the output layer 26 are not shown.

  Hereinafter, the provisional output layer addition (step S3) will be described in detail by taking as an example the case where the intermediate layers 22 and 23 are selected in step S2 (see FIG. 3). First, the temporary output layer adding unit 13 identifies the intermediate layer 23 positioned in the upward direction among the selected intermediate layers 22 and 23. The temporary output layer adding unit 13 adds a temporary output layer 33 corresponding to the identified intermediate layer 23 to the neural network 3A.

  Specifically, the temporary output layer adding unit 13 generates a temporary output layer 33 having temporary output nodes 33a to 33c. The temporary output layer adding unit 13 adds the temporary output layer 33 to the neural network 3A by connecting each of the temporary output nodes 33a to 33c to the nodes 23a to 23d of the intermediate layer 23. The number of temporary output nodes included in the temporary output layer 33 is set to be the same as the number of output nodes included in the output layer 26. The reason for this will be described later.

  The temporary output layer adding unit 13 sets the initial value of the weight count of each signal path in the section G <b> 1 between the intermediate layer 23 and the temporary output layer 33 with the addition of the temporary output layer 33. The initial value of the weight count is set at random, for example, so as to be within a predetermined dispersion range.

  Next, the temporary output layer adding unit 13 identifies the intermediate layer 22 positioned in the downward direction among the selected intermediate layers 22 and 23. The temporary output layer adding unit 13 adds the temporary output layer 32 corresponding to the identified intermediate layer 22 to the neural network 3A.

  Specifically, the temporary output layer adding unit 13 generates a temporary output layer 32 having temporary output nodes 32a to 32c. The temporary output layer adding unit 13 adds the temporary output layer 32 to the neural network 3 </ b> A by connecting each of the temporary output nodes 32 a to 32 c to the nodes 22 a to 22 d of the intermediate layer 22. The number of temporary output nodes included in the temporary output layer 32 matches the number of output nodes included in the output layer 26.

  The temporary output layer adding unit 13 sets the initial value of the weight count of each signal path in the section F <b> 1 between the intermediate layer 22 and the temporary output layer 32 with the addition of the temporary output layer 32. The initial value of the weight count is randomly set so as to be within a predetermined dispersion range.

  Each of the temporary output nodes 32a to 32c included in the temporary output layer 32 corresponds to any one of the output nodes 26a to 26c. The same applies to the temporary output nodes 33a to 33c included in the temporary output layer 33. Specifically, the temporary output nodes 32a and 33a correspond to the output node 26a. The temporary output nodes 32b and 33b correspond to the output node 26b. The temporary output nodes 32c and 33c correspond to the output node 26c.

  In this way, the temporary output layers 32 and 33 are added to the neural network 3A. The temporary output layer adding unit 13 outputs a network obtained by adding temporary output layers 32 and 33 to the neural network 3A as a neural network 3B.

{3.1.4. Additional learning}
Reference is again made to FIG. The additional learning unit 14 inputs the neural network 3B output from the temporary output layer adding unit 13. The additional learning unit 14 performs additional learning of the neural network 3B (step S4).

  The additional learning (step S4) will be described in detail with reference to FIG. The additional learning unit 14 updates only the weighting coefficient of the signal path in the sections F1 and G1. The weighting factor of the signal path in the sections A1, B1, C1, D1, and E1 is not updated. That is, the additional learning (step S4) is a process of updating the signal path weight count in the section between the intermediate layer selected in step S2 and the temporary output layer connected to the selected intermediate layer.

  Additional learning is performed using the error back propagation method. Specifically, the additional learning unit 14 inputs the learning data 4 to the neural network 3B, and executes a calculation using the neural network 3B. The learning data 4 is, for example, image data obtained by photographing any of a dog, a cat, and other objects.

  The additional learning unit 14 obtains temporary output values 42a to 42c and 43a to 43c by calculation of the neural network 3B with the learning data 4 as an input.

  The temporary output values 42a to 42c are output values output from the temporary output nodes 32a to 32c by the calculation of the neural network 3B when learning data 4 or test data 5 described later is input to the neural network 3B. Hereinafter, the temporary output values 42 a to 42 c are collectively referred to as the temporary output value 42. The temporary output values 43a to 43c are output values output from the temporary output nodes 33a to 33c by the calculation of the neural network 3B. Hereinafter, the temporary output values 43a to 43c are collectively referred to as the temporary output value 42. Hereinafter, the temporary output values 42 and 43 obtained by inputting the learning data 4 to the neural network 3B are described as temporary output values for additional learning.

  The additional learning unit 14 calculates an error value corresponding to each of the temporary output nodes 32a to 32c by subtracting the temporary output value 42 for additional learning from the correct value. The correct answer value (teacher signal) is a value determined in advance as a value to be output by each of the output nodes 26a to 26c when the learning data 4 is input to the neural network 3B.

  For example, the error value corresponding to the temporary output node 32a is obtained by subtracting the temporary output value 42a for additional learning from the correct value of the output node 26a. The additional learning unit 14 inputs an error value corresponding to each of the temporary output nodes 32a to 32c to the temporary output nodes 32a to 32c, and updates the weighting coefficient of the signal path in the section F1 based on the input error value.

  Similarly, the additional learning unit 14 subtracts the temporary output value 43 for additional learning from the correct value, thereby calculating an error value corresponding to each of the temporary output nodes 33a to 33c. The additional learning unit 14 inputs an error value corresponding to each of the temporary output nodes 33a to 33c to the temporary output nodes 33a to 33c, and updates the weighting coefficient of the signal path in the section G1 based on the input error value.

  The reason why only the weighting factor of the signal path in the sections F1 and G1 is updated in the additional learning will be described. When performing additional learning (step S4), the additional learning unit 14 can update the weighting factor of the signal path in the section A1 based on the error value input to each of the temporary output nodes 32a to 32c. It is. Further, the additional learning unit 14 can update the weighting factor of the signal path in each of the sections A1 and B1 based on the error value input to each of the temporary output nodes 33a to 33c.

  However, since the neural network 3A has already been learned, the weighting factor of the signal path in each of the sections A1 and B1 has already been adjusted. In the additional learning (step S4), when the signal path weight count in each of the sections A1 and B1 is updated, the learning accuracy of the neural network 3A may be lowered. For this reason, the additional learning unit 14 updates only the weighting coefficient of the signal path in the sections F1 and G1 provided by adding the temporary output layers 32 and 33. The neural network 3B for which additional learning has been completed is output to the arithmetic unit 15 as the neural network 3C.

{3.1.5. Calculation of similarity}
The calculation unit 15 receives the neural network 3C output from the additional learning unit 14. The calculation unit 15 inputs the test data 5 to the neural network 3C, and executes a calculation using the neural network 3C (step S5). In the present embodiment, the test data 5 is image data obtained by photographing any of a dog, a cat, and other objects. As shown in FIG. 4, the neural network 3C outputs the temporary output value 42 from the temporary output nodes 32a to 32c and outputs the temporary output value 43 from the temporary output nodes 33a to 33c as the calculation results using the test data 5. To do.

  The deletion determination unit 16 acquires the temporary output values 42 and 43 from the calculation unit 15. The deletion determining unit 16 calculates the similarity between the intermediate layer 22 and the intermediate layer 23 using the acquired temporary output values 42 and 43 (step S6). The degree of similarity is obtained by calculating a batcha rear distance, which is a parameter indicating the similarity between two probability density distributions. Specifically, the similarity is calculated using the following formula (1).

  In equation (1), S is the similarity. p (i) is a probability density distribution of temporary output values output from the temporary output node in one temporary output layer of the two temporary output layers. q (i) is a probability density distribution of temporary output values output from the temporary output node in the other temporary output layer. i indicates the i-th node in each temporary output layer.

  In the case of the example shown in FIG. 4, p (i) can be assigned to the probability density distribution of the temporary output value 42, and q (i) can be assigned to the probability density distribution of the temporary output value 43. The deletion determination unit 16 normalizes each of the temporary output values 42a to 42c to a numerical value of 0 or more and 1 or less, and creates a probability density distribution p (i) using the normalized temporary output values 42a to 42c. To do. The probability density distribution q (i) of the temporary output value 43 is created in the same manner.

  The similarity calculated by the expression (1) is a numerical value of 0 or more and 1 or less. As the difference between the temporary output value 42 and the temporary output value 43 decreases, the degree of similarity approaches 1. One similarity is calculated for one test data 5. The machine learning device 100 calculates the similarity corresponding to each of the plurality of test data 5 by repeating steps S5 and S6.

  In step S5, the neural network 3C outputs a normal output value 46 as a calculation result of the test data 5. However, when the intermediate layers 22 and 23 are selected in step S2, the normal output value 46 is not used for calculating the similarity. An example in which the normal output value 46 is used for calculating the similarity will be described later.

{3.1.6. Deletion judgment}
As described above, the calculation unit 15 inputs a predetermined number of test data 5 to the neural network 3C and calculates the neural network 3C, so that the deletion determination unit 16 calculates a predetermined number of similarities (steps) S5 and S6).

  When the calculation of the neural network 3C using the predetermined number of test data 5 is completed, the deletion determining unit 16 calculates an average value of the predetermined number of similarities. The deletion determining unit 16 determines whether any one of the intermediate layers 22 and 23 can be deleted by comparing the calculated average value with a preset similarity reference value (step S7).

  As described above, the similarity degree approaches 1 as the difference between the temporary output value 42 and the temporary output value 43 decreases. As the average similarity value approaches 1, it is determined that the functions of the intermediate layers 22 and 23 in the neural network 3A are similar.

  The deletion determining unit 16 determines that the function of the intermediate layer 22 is similar to the function of the intermediate layer 23 when the average value of the similarities is larger than the similarity reference value. In this case, the deletion determination unit 16 determines that any one of the intermediate layers 22 and 23 can be deleted (Yes in step S7). The deletion determination unit 16 determines to delete the intermediate layer 22 positioned in the downward direction among the intermediate layers 22 and 23. The intermediate layer 22 is located farther from the output layer 26 than the intermediate layer 23. For this reason, the influence of the intermediate layer 22 on the normal output value 46 output from the output nodes 26 a to 26 c of the output layer 26 is smaller than that of the intermediate layer 23. For this reason, the intermediate layer 22 located in the lower direction is deleted from the two selected intermediate layers 22 and 23.

  Here, the machine learning device 100 calculates the similarity from the temporary output values output from the two temporary output layers, and determines whether the two intermediate layers are similar using the calculated similarity. Explain why. In determining whether two intermediate layers are similar, it is conceivable to compare the output value from one intermediate layer with the output value from the other intermediate layer. However, even if the output value from one intermediate layer is compared with the output value from the other intermediate layer, it is difficult to determine whether or not the two intermediate layers are similar. There are two reasons why it is difficult.

  The first reason is that the number of nodes included in each of the two selected intermediate layers is not necessarily the same. In the example shown in FIG. 4, the number of each of the intermediate layers 22 and 33 selected in step S <b> 2 is the same. However, if each of the two selected intermediate layers has a different number of nodes, the output value from the node of one intermediate layer cannot be simply compared with the output value from the node of the other intermediate layer. .

  The second reason is that even if the functions of the two selected intermediate layers are similar and the number of nodes included in the two intermediate layers is the same, the output value from the node included in one of the intermediate layers This is because the value is not always similar to the output value from the node of the other intermediate layer. This will be described in detail below.

  When the error back-propagation method is used at the time of learning of the neural network 3A shown in FIG. 4, the initial values of the signal path weight coefficients in the sections A1 and B1 are randomly set so as to fall within a predetermined dispersion range. Further, since the error propagating downward in the section A1 is smaller than the error propagating downward in the section B1, the update amount of the weighting coefficient of the signal path in the section A1 is the update of the weighting coefficient of the signal path in the section B1. Different from the amount.

  As a result, even when the intermediate layers 22 and 23 have similar functions, the weighting coefficients of the two signal paths that are in a correspondence relationship in the sections A1 and B1 are not the same. For example, in FIG. 4, the signal path between the node 21b and the node 22a in the section A1 corresponds to the signal path between the node 22b and the node 23a in the section B1. However, the weighting factors of these two signal paths are not the same. Even if the value output from the node 22a matches the value output from the node 23a corresponding to the node 22a, it cannot be determined that the functions of the selected intermediate layers 22 and 23 are similar.

  Therefore, the machine learning device 100 connects the temporary output layer to each of the two selected intermediate layers, and the functions of the two selected intermediate layers are similar based on the similarity of the output values from the two temporary output layers. Judge whether to do.

  The number of nodes included in the temporary output layer connected to one intermediate layer is made equal to the number of nodes included in the temporary output layer connected to the other intermediate layer. In addition, each node included in the two temporary output layers is made to correspond one-to-one with a node included in the output layer. As a result, it is possible to clarify the correspondence between a node included in the temporary output layer connected to one intermediate layer and a node included in the temporary output layer connected to the other intermediate layer.

  The weight count in the signal path between one intermediate layer and the temporary output layer connected to one intermediate layer is updated by additional learning (step S4). In the additional learning, the error input to the temporary output node is calculated from the correct value to be output from the output nodes 26a to 26c. Therefore, when the test data 5 is input to the neural network 3B, the temporary output value output from the temporary output layer includes the output value output from each node of one intermediate layer and the normal output output from the output node. This corresponds to a parameter indicating the relationship with the value 46. That is, the temporary output value output from the node included in the temporary output layer can be used as a parameter reflecting the function of the intermediate layer to which the temporary output layer is connected.

  Specifically, the temporary output value 42 is a parameter reflecting the function of the intermediate layer 22 because it is calculated based on the output value of the intermediate layer 22. The temporary output value 43 is a parameter reflecting the function of the intermediate layer 23 because it is calculated based on the output value of the intermediate layer 23. Further, since the correspondence relationship between the temporary output nodes 23a to 23c included in the temporary output layer 32 and the temporary output nodes 33a to 33c included in the temporary output layer 33 is clear, the temporary output values 42a to 42c and the temporary output The correspondence with the values 43a to 43c is also clear. Therefore, by using the similarity calculated from the probability density distribution of the temporary output value 42 and the probability density distribution of the temporary output value 43, it is determined whether or not the two selected intermediate layers 22 and 23 are similar. It becomes possible to judge.

{3.1.7. Reconfiguration of neural network 3C}
The reconfiguration unit 17 reconfigures the neural network 3A when the deletion determination unit 16 determines to delete the intermediate layer located in the lower direction of the two intermediate layers (step S8).

  For example, when it is determined that the intermediate layer 22 is to be deleted from the intermediate layers 22 and 23, the reconfiguration unit 17 releases the connection between the node included in the input layer 21 and the node included in the intermediate layer 22 in the neural network 3A. The connection between the node included in the intermediate layer 22 and the node included in the intermediate layer 23 is released. Then, the reconfiguration unit 17 sets a new signal path by connecting the node included in the input layer 21 and the node included in the intermediate layer 23. Thereby, the reconfiguration unit 17 generates a neural network 3D as a network in which the intermediate layer 22 is deleted from the neural network 3A.

  FIG. 5 is a diagram showing a configuration of the neural network 3D generated in step S8. As shown in FIG. 5, the neural network 3 </ b> D does not include the intermediate layer 22, unlike the neural network 3 </ b> A shown in FIG. 2. In the neural network 3D, input nodes 21a to 21d included in the input layer 21 are directly connected to nodes 23a to 23d included in the intermediate layer 23. Hereinafter, a section between the input layer 21 and the intermediate layer 23 is referred to as a section A2.

  The signal path in the section A2 is newly set by the reconstruction unit 17. The reconstruction unit 17 sets the initial value of the weighting factor of the signal path in the section A2 so as to be within a predetermined dispersion range. Thereafter, the reconfiguration unit 17 performs relearning of the neural network 3D (step S9).

  For the relearning, learning data used in the learning of the neural network 3A is used. The re-learning algorithm is the same as the algorithm used for learning of the neural network 3A. By re-learning, not only the signal path weight coefficient in the section A2 but also the signal path weight coefficient in the sections C1, D1, and E1 are updated in the neural network 3D. Thereby, the neural network 3D can have the same learning accuracy as the neural network 3A.

{3.1.8. End judgment}
When the re-learning of the neural network 3D by the reconfiguration unit 17 ends, the end determination unit 18 determines whether or not to end the process of deleting the intermediate layer (step S10).

  When the number of intermediate layers deleted from the neural network 3A reaches a preset reference end value, the end determination unit 18 determines to end the process of deleting the intermediate layer. The number of deleted intermediate layers is obtained by deleting the number of intermediate layers included in the neural network 3B from the number of intermediate layers included in the neural network 3A. When the number of intermediate layers exceeding the end reference value is deleted from the neural network 3A, the scale of the neural network 3D is significantly smaller than the scale of the neural network 3A input to the machine learning device 100. As a result, the learning accuracy of the neural network 3D may be adversely affected. The end determination unit 18 determines whether or not to continue the deletion of the intermediate layer based on the number of deleted intermediate layers in order to prevent a decrease in the learning accuracy of the neural network 3D.

  At present, only the intermediate layer 22 is deleted from the neural network 3A, and the number of deleted intermediate layers is one. When the end reference value is set to 2, the number of deleted intermediate layers has not reached the end reference value (No in step S10). For this reason, the termination determination unit 18 continues the process of deleting the intermediate layer from the neural network 3A. Specifically, the end determination unit 18 outputs the re-learned neural network 3D to the intermediate layer selection unit 12 and the temporary output layer addition unit 13. The machine learning device 100 returns to step S2 again and executes a process of deleting the intermediate layer from the neural network 3D.

{3.2. Delete intermediate layer 24}
{3.2.1. Select intermediate layer and add temporary output layer}
The intermediate layer selection unit 12 and the temporary output layer addition unit 13 acquire the neural network 3D from the end determination unit 18. The intermediate layer selection unit 12 selects two intermediate layers from the intermediate layers that were not selected at the previous selection in the neural network 3D (step S2). Specifically, the intermediate layers 22 and 23 are selected at the previous selection. The intermediate layer selection unit 12 selects the intermediate layers 24 and 25 among the intermediate layers 23 to 25 included in the neural network 3D.

  The temporary output layer adding unit 13 adds a new temporary output layer to the neural network 3D based on the selection by the intermediate layer selecting unit 12 (step S3). FIG. 6 is a diagram showing a neural network 3E that is a network in which a temporary output layer is added to the neural network 3D. In the neural network 3E shown in FIG. 6, the display of the input layer 21 and the intermediate layer 22 is omitted.

  As illustrated in FIG. 6, the temporary output layer adding unit 13 connects the temporary output layer 34 only to the intermediate layer 24 among the intermediate layers 24 and 25 selected by the intermediate layer selecting unit 12. The temporary output layer adding unit 13 does not connect the temporary output layer to the intermediate layer 25. The intermediate layer 25 selected by the intermediate layer selector 12 is adjacent to the output layer 26. Therefore, it is determined that the output layer 26 can be used as the temporary output layer connected to the intermediate layer 25, and the temporary output layer is not connected to the intermediate layer 25.

  As illustrated in FIG. 6, the temporary output layer 34 includes the same number of nodes as the nodes included in the output layer 26. The temporary output layer 34 includes temporary output nodes 34a to 34c. The temporary output node 34a corresponds to the output node 26a. The temporary output node 34b corresponds to the output node 26b. The temporary output node 34c corresponds to the output node 26c.

  Each of temporary output nodes 34a to 34 is connected to nodes 24a to 24d of intermediate layer 24. The temporary output layer adding unit 13 sets an initial value of the signal path weight count in the section H <b> 1 between the intermediate layer 24 and the temporary output layer 34 with the addition of the temporary output layer 34. Similar to the addition of the temporary output layers 32 and 33, the initial value of the weight count is set at random so as to be within a preset range of dispersion.

{3.2.2. Additional learning}
The additional learning unit 14 receives the neural network 3E output from the temporary output layer adding unit 13. The additional learning unit 14 performs additional learning of the neural network 3E (step S4). By additional learning of the neural network 3E, the signal path weight count in the section H1 is updated. In the additional learning of the neural network 3E, the weighting coefficient of the signal path in the sections A1, C1, D1, and E1 is not updated. Since the additional learning of the neural network 3E is the same as the additional learning of the neural network 3B, detailed description thereof is omitted.

{3.2.3. Calculation of similarity}
The calculation unit 15 receives the neural network 3E output from the additional learning unit 14. The calculation unit 15 inputs the test data 5 to the acquired neural network 3E, and executes a calculation using the neural network 3E (step S5).

  As a calculation result, the neural network 3E outputs temporary output values 44a to 44c from the temporary output nodes 34a to 34c, and outputs normal output values 46a to 46c from the output nodes 26a to 26c. Hereinafter, the temporary output values 44a to 44c are collectively referred to as the temporary output value 44.

  The deletion determination unit 16 acquires the temporary output value 44 and the normal output value 46 from the calculation unit 15. The deletion determination unit 16 calculates the similarity between the intermediate layers 24 and 25 selected by the intermediate layer selection unit 12 using the acquired temporary output value 44 and normal output value 46 (step S6). In the above equation (1), by assigning p (i) to the probability density distribution of the provisional output value 44 and assigning q (i) to the probability density distribution of the normal output value 46, the similarity of the intermediate layers 24 and 25 is obtained. Can be calculated.

  The calculation unit 15 inputs a predetermined number of test data 5 to the neural network 3E, and executes a calculation using the neural network 3E (step S5). The deletion determining unit 16 acquires the temporary output value 44 and the normal output value 46 corresponding to each test data 5, and calculates the similarity corresponding to each test data 5 (step S6).

{3.2.4. Deletion judgment}
When the calculation of the neural network 3E using the predetermined number of test data 5 is completed, the deletion determining unit 16 calculates an average value of the plurality of similarities calculated, and compares the calculated similarity with a deletion reference value. Here, it is assumed that the similarity between the intermediate layers 24 and 25 is larger than the deletion reference value. In this case, the deletion determination unit 16 determines that any one of the intermediate layers 24 and 25 can be deleted (Yes in step S7). The deletion determining unit 16 determines to delete the intermediate layer 24 positioned in the downward direction among the intermediate layers 24 and 25.

{3.2.5. Reconfiguration of neural network}
The reconfiguration unit 17 reconfigures the neural network 3D based on the determination of the deletion of the intermediate layer 24 by the deletion determination unit 16 (step S8). Specifically, the reconfiguration unit 17 releases the connection between the nodes 23a to 23d included in the intermediate layer 23 and the nodes 24a to 24d included in the intermediate layer 24 in the neural network 3D, and the node 24a included in the intermediate layer 24. To 24d and the nodes 25a to 25d included in the intermediate layer 25 are disconnected. The reconfiguration unit 17 connects each of the nodes 23a to 23d included in the intermediate layer 23 to the nodes 25a to 25d included in the intermediate layer 25. As a result, the intermediate layer 24 is deleted from the neural network 3D.

  FIG. 7 is a diagram showing a neural network 3F that is a network obtained by deleting the intermediate layer 24 from the neural network 3D shown in FIG. As shown in FIG. 7, the neural network 3F does not include the intermediate layer 24, unlike the neural network 3D shown in FIG. In the neural network 3F, each of the nodes 23a to 23d included in the intermediate layer 23 is directly connected to the nodes 25a to 25d included in the intermediate layer 25. Hereinafter, a section between the intermediate layer 23 and the intermediate layer 25 in the neural network 3F is referred to as a section C2.

  The signal path in the section C2 is newly set by the reconstruction unit 17. The reconstruction unit 17 sets the initial value of the weighting factor of the signal path in the section C2 so as to be within a predetermined dispersion range. Thereafter, the reconfiguration unit 17 performs relearning of the neural network 3F (step S9). By re-learning, not only the signal path weighting coefficient in the section C2 but also the signal path weighting coefficients in the sections A2 and E1 are updated in the neural network 3F. Since the relearning of the neural network 3F is the same as the relearning of the neural network 3D, detailed description thereof is omitted.

  When the re-learning of the neural network 3F by the reconfiguration unit 17 ends, the end determination unit 18 determines whether or not to end the process of deleting the intermediate layer from the neural network 3A (step S10).

  At this time, since the intermediate layers 22 and 24 are deleted from the neural network 3A, the number of deleted intermediate layers is two. When the end reference value is set to 2, the number of deleted intermediate layers has reached the reference value (Yes in step S10). In this case, the end determination unit 18 determines that the process of deleting the intermediate layer from the neural network 3A should be ended. The machine learning device 100 outputs the re-learned neural network 3F.

  Further, when all the intermediate layers included in the neural network 3A are selected, the end determination unit 18 ends the process of deleting the intermediate layer from the neural network 3A. This is because when all the intermediate layers are selected, the machine learning device 100 cannot newly specify an intermediate layer that can be deleted.

  In this way, the machine learning device 100 selects two intermediate layers in the input neural network 3A, and connects the temporary output layer to each of the two selected intermediate layers. The machine learning device 100 includes a temporary output value output from a temporary output layer connected to one of the two selected intermediate layers, and a temporary output output from a temporary output layer connected to the other intermediate layer. Based on the value, it is determined whether one of the two selected intermediate layers can be deleted. When it is determined that the machine learning device can be deleted, the machine learning device generates a network in which one of the two selected intermediate layers is deleted from the neural network 3A. Thereby, the machine learning device 100 can create a neural network 3E having learning accuracy equivalent to that of the neural network 3A and having a smaller scale than the neural network 3A. In other words, the machine learning device 100 can create a neural network having a learning accuracy required in advance and having an appropriate scale.

{Modifications}
In the above embodiment, the example in which the intermediate layer selection unit 12 selects the intermediate layers 24 and 25 from the neural network 3D (see FIG. 5) has been described, but the present invention is not limited to this. The intermediate layer selection unit 12 may select the intermediate layer 23 again in the neural network 3D. Specifically, the intermediate layer selection unit 12 may select the intermediate layers 23 and 24 when the neural network 3D is input. If the similarity between the intermediate layers 23 and 24 exceeds the deletion reference value, the intermediate layer 23 is deleted from the neural network 3D.

  In the said embodiment, although the intermediate | middle layer selection part 12 demonstrated the example selected in order from the intermediate | middle layer located in a downward direction, it is not restricted to this. The order in which the intermediate layer selection unit 12 selects the intermediate layer is not particularly limited, and may be selected in order from the intermediate layer positioned in the upward direction.

  In the above embodiment, the end determination unit 18 determines whether to end the process of deleting the intermediate layer from the neural network 3A based on whether the number of deleted intermediate layers has reached the end reference value. Although the example to judge was demonstrated, it is not restricted to this. The end determination unit 18 may determine to end the process of deleting the intermediate layer when the number of intermediate layers in the neural network output from the reconfiguration unit 17 reaches a preset reference value. For example, consider a case where the reference value is set to 3. In this case, since the number of intermediate layers included in the neural network 3D illustrated in FIG. 6 is 3, the end determination unit 18 does not execute the process of deleting the intermediate layer 24 from the neural network 3D. The end determination unit 18 outputs the neural network 3D to the outside of the machine learning device 100.

  In the above embodiment, an example has been described in which the deletion determination unit 16 calculates the similarity between two intermediate layers using Expression (1), but the present invention is not limited to this. The deletion determining unit 16 normalizes each value of the temporary output values 42a to 42c to a numerical value of 0 or more and 1 or less, and similarly normalizes each value of the temporary output values 43a to 43c. Then, the deletion determining unit may calculate a difference absolute value between two corresponding temporary output values, and calculate the sum of the difference absolute values as the similarity. In this case, the degree of similarity indicates that the functions of the two intermediate layers are similar as it approaches zero.

  In addition, part or all of the processing of each functional block (each functional unit) of the machine learning device 100 in the above embodiment may be realized by a program. In the machine learning device 100 of the above embodiment, part or all of the processing of each functional block is performed by a central processing unit (CPU) in the computer. In addition, a program for performing each processing is stored in a storage device such as a hard disk or a ROM, and is read out and executed in the ROM or the RAM. For example, by configuring the machine learning device 100 as shown in FIG. 13, a part or all of the processing of each functional block (each functional unit) in each of the above embodiments is executed. Also good.

  In addition, each process of the above embodiment may be realized by hardware, or may be realized by software (including a case where it is realized together with an OS (operating system), middleware, or a predetermined library). . Further, it may be realized by mixed processing of software and hardware.

  Moreover, the execution order of the processing method in the said embodiment is not necessarily restricted to description of the said embodiment, The execution order can be changed in the range which does not deviate from the summary of invention.

  A computer program that causes a computer to execute the above-described method and a computer-readable recording medium that records the program are included in the scope of the present invention. Here, examples of the computer-readable recording medium include a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, large-capacity DVD, next-generation DVD, and semiconductor memory. .

  The computer program is not limited to the one recorded on the recording medium, and may be transmitted via a telecommunication line, a wireless or wired communication line, a network represented by the Internet, or the like.

  In addition, the word “part” may be a concept including “circuitry”. The circuit may be realized in whole or in part by hardware, software, or a mixture of hardware and software.

  While the embodiments of the present invention have been described above, the above-described embodiments are merely examples for carrying out the present invention. Therefore, the present invention is not limited to the above-described embodiment, and can be implemented by appropriately modifying the above-described embodiment without departing from the spirit thereof.

DESCRIPTION OF SYMBOLS 100 Machine learning apparatus 11 Network acquisition part 12 Intermediate | middle layer selection part 13 Temporary output layer addition part 14 Additional learning part 15 Calculation part 16 Deletion judgment part 17 Reconstruction part 18 End judgment part

Claims (8)

An acquisition unit for acquiring a learned neural network including at least two intermediate layers;
An intermediate layer selection unit for selecting a first intermediate layer and a second intermediate layer from the at least two intermediate layers;
A first temporary output layer having a temporary output node connected to a node of the first intermediate layer; and a second temporary output layer having a temporary output node connected to a node of the second intermediate layer. A temporary output layer adding unit for generating a temporary network in addition to the network;
The test data is input to the temporary network, the calculation using the temporary network is executed, the first temporary output value output from the temporary output node of the first temporary output layer, and the second temporary output layer An arithmetic unit for obtaining a second temporary output value output from the temporary output node having;
The first temporary output value and the second temporary output value are used to calculate a similarity indicating the degree to which the function of the first intermediate layer is similar to the function of the second intermediate layer, and based on the calculated similarity A deletion determination unit for determining whether to delete the one intermediate layer;
A reconfiguration unit that deletes the first intermediate layer from the neural network when the deletion determination unit determines to delete the first intermediate layer;
A machine learning device comprising: The machine learning device according to claim 1,
The machine learning device, wherein the first intermediate layer is disposed closer to an input layer included in the neural network than the second intermediate layer. The machine learning device according to claim 1 or 2,
The machine learning device, wherein the number of temporary output nodes included in the first temporary output layer is the same as the number of temporary output nodes included in the second temporary output layer. The machine learning device according to claim 3,
The machine learning device, wherein the number of temporary output nodes included in the first temporary output layer is the same as the number of output nodes included in the output layer in the neural network. The machine learning device according to any one of claims 1 to 4,
The provisional output layer adding unit determines to use the output layer as the second provisional output layer when a node of the second intermediate layer is connected to a node of the output layer in the neural network. Machine learning device. The machine learning device according to any one of claims 1 to 5, further comprising:
A weighting factor set in a signal path between a node included in the first intermediate layer and a node included in the first temporary output layer; a node included in the second intermediate layer; and a node included in the second temporary output layer. An additional learning unit that performs learning to update the weighting factor set in the signal path between
A machine learning device comprising: The machine learning device according to any one of claims 1 to 6, further comprising:
When the first intermediate layer is a kth (k is a natural value between 2 and m−1) counting from the input layer of the neural network, the (k−1) th layer is And the node of the (k + 1) th layer is newly connected to the node of the (k + 1) th layer and the signal path between the node of the (k-1) th layer and the node of the (k + 1) th layer is set. A machine learning device that performs learning for updating a weighting coefficient. Obtaining a trained neural network comprising at least two intermediate layers;
Selecting a first intermediate layer and a second intermediate layer from the at least two intermediate layers;
A first temporary output layer having a temporary output node connected to a node of the first intermediate layer; and a second temporary output layer having a temporary output node connected to a node of the second intermediate layer. Adding to the network and generating a temporary network;
The test data is input to the temporary network, the calculation using the temporary network is executed, the first temporary output value output from the temporary output node of the first temporary output layer, and the second temporary output layer Obtaining a second temporary output value output from the temporary output node having;
The first temporary output value and the second temporary output value are used to calculate a similarity indicating the degree to which the function of the first intermediate layer is similar to the function of the second intermediate layer, and based on the calculated similarity Determining whether to delete the one intermediate layer;
If it is determined to delete the first intermediate layer, deleting the first intermediate layer from the neural network;
A machine learning method comprising:

Images