JP7316771B2 - Learning device, parameter creation method, neural network, and information processing device using the same - Google Patents

Description

The present invention relates to a learning device, a method of creating parameters, a neural network, and an information processing device using the same, and more particularly to image recognition technology, for example.

A method of generating a discriminator for discriminating input data by performing hierarchical network learning using learning data is known. On the other hand, as the number of layers in a hierarchical network increases, the so-called vanishing gradient problem (deltas disappear or diverge when backpropagating the deltas needed to update the weighting coefficients) becomes apparent, impeding the progress of learning. known to occur.

As a method for coping with such a problem, a method called deep supervision, in which error evaluation and error backpropagation are performed even in the middle layer of the network (hereinafter referred to as side-out learning), is known (non-patent literature 1). In addition to training the hierarchical network to extract the feature values of an image, we also learned to activate specific neurons when specific features are present. There is also known a method that enables feature quantity extraction (Patent Document 1).

JP 2016-31746 A

Xie, S., Tu, Z. "Holistically-nested edge detection" ICCV, 1395-1403 (2015)

However, in the method of Non-Patent Document 1, it was found that the error evaluation accuracy for the output from the intermediate layer is low, and there is a possibility that a favorable final learning result cannot be obtained.

SUMMARY OF THE INVENTION It is an object of the present invention to make neural network learning more effective.

In order to achieve the object of the present invention, for example, the learning device of the present invention has the following configuration. i.e.
A learning device for learning a neural network used for attribute determination processing of each pixel of an image ,
setting means for setting teacher data for each of two or more different outputs from the neural network corresponding to a single input data for learning;
Learning means for learning the neural network based on the error between each of the two or more different outputs obtained by inputting the learning input data to the neural network and teacher data corresponding to the output. and
The training data for each of the two or more different outputs indicates the results of different deformation processing or filtering processing on the basic training data corresponding to the single learning input data, and the basic training data is the single learning input data. is image data indicating the attribute of each pixel indicated by the input data for learning .

Neural network learning can be performed more effectively.

1 is a functional configuration diagram showing an example of a learning device according to an embodiment; FIG. 4 is a flowchart illustrating an example of a parameter generation method according to one embodiment; Schematic diagram showing an example of a hierarchical network that performs side-out learning. FIG. 4 is a diagram for explaining the relationship between side-out output and GT; FIG. 10 is a diagram for explaining a problem when side-out learning is performed according to the conventional technology; FIG. 4 is a diagram for explaining a method of generating adaptive GTs according to one embodiment; FIG. 4 is a diagram for explaining a method of generating adaptive GTs according to one embodiment; FIG. 4 is a diagram for explaining a method of generating adaptive GTs according to one embodiment; FIG. 4 is a diagram for explaining a method of generating adaptive GTs according to one embodiment; FIG. 4 is a diagram for explaining a method of generating adaptive GTs according to one embodiment; 1 is a schematic block diagram of a computer used in one embodiment; FIG.

Hereinafter, embodiments of the present invention will be specifically described with reference to flowcharts and drawings. In addition, although the following specific examples are examples of embodiments according to the present invention, the present invention is not limited to the following specific forms. INDUSTRIAL APPLICABILITY The present invention can be applied to learning a hierarchical network (hereinafter sometimes referred to as a neural network or simply a network) using learning data, and can be applied to any method for learning a hierarchical network. be.

FIG. 1 shows an example of the functional configuration of a learning device 100 according to the first embodiment. The learning device 100 learns a hierarchical network. The basic data storage unit 101 holds basic learning data used for learning. Learning data is teacher data (hereinafter sometimes referred to as GT) used for network learning. In the present embodiment, learning of the network is performed using input data for learning and teacher data (learning data) indicating determination results for the input data for learning.

For example, in one embodiment, a network can be used to perform attribute determination (labeling) for each pixel of an image. That is, when image data is input to the network as input data, attribute information (label) of each pixel of the image data is obtained as a result of determination processing on the input data. For example, in a specific example of extracting the contour of an image, as a result of determination processing for input data, a contour pattern (an image having attribute information indicating whether or not it is a contour as a pixel value) corresponding to the input data is obtained. can get. Thus, in one embodiment, the result of processing the input data may be a line drawing pattern (such as a contour pattern) corresponding to the input data.

In such a configuration, the input data for learning is image data, and the learning data is data indicating a label (determination result) for each pixel of the input data for learning. For example, the input data for learning can be an image including characters, graphics, or the like. In a specific example of extracting the contour of an image, the learning data is an image showing the contour in the image, which is the input data for learning, and can be created according to user input, for example. The basic data storage unit can further hold such learning input data in combination with learning data. In this specification, basic learning data (basic teacher data) refers to learning data (teacher data) before processing such as processing or transformation by the setting unit 102 is performed.

The setting unit 102 sets learning data used for network learning. Also, the adaptive data storage unit 103 holds learning data set by the setting unit 102 . In one embodiment, the setting unit 102 generates learning data by processing, transforming, or filtering the basic learning data. The setting unit 102 stores the learning data generated in this way in the adaptive data storage unit 103 to obtain learning data used for network learning (hereinafter referred to as adaptive learning data, adaptive teacher data, or adaptive GT). (sometimes called). Setting section 102 may further store the original basic learning data in adaptive data storage section 103 . As will be described later, the setting unit 102 sets learning data (adaptive teacher data) for each of two or more different outputs from the hierarchical network corresponding to single learning input data.

The learning unit 104 reads learning data stored in the adaptive data storage unit 103 and performs network learning processing. Also, the learning unit 104 stores the final learning result (for example, network parameters) obtained by learning in the learning result storage unit 105 . A known method can be used as a learning method for the hierarchical network. For example, by back-propagating in this network the errors in the output values resulting from the forward propagation calculations in the hierarchical network, the weighting factors and other parameters corresponding to the connectivity states of the network can be iteratively updated. As will be described later, in the present embodiment, the learning unit 104 includes two or more different outputs obtained by inputting learning input data to a network, learning data corresponding to the outputs (adaptive teacher data), The hierarchical network is trained based on the error of

The test data storage unit 106 holds test data used for network evaluation. The evaluation unit 107 evaluates the network using test data. The hierarchical network after learning thus obtained provides two or more different outputs corresponding to the input data, as will be described later. The integration result of two or more different outputs obtained in this manner indicates the result of recognition processing for the input data.

FIG. 2 is a flow chart of the learning method according to this embodiment. Description will be made below along this flow chart. In step S<b>210 , the setting unit 102 reads basic learning data from the basic data storage unit 101 . In step S220, the setting unit 102 sets adaptive learning data based on the basic learning data. Here, the setting unit 102 sets learning data (adaptive teacher data) for each of two or more different outputs based on the structure of the hierarchical network. The adaptive learning data to be set can depend on the configuration of the units forming the hierarchical network or their connection state. In the following, as an example, a case of performing side-out learning (a method of performing learning based not only on the output error of the final layer but also on the output error of the intermediate layer, the details of which will be described later) will be described.

In step S230, the learning unit 104 performs hierarchical network learning using the adaptive learning data set in step S220. The network used in this embodiment has a side-out (output from the intermediate layer), and the determination result can be obtained based on this side-out. A specific learning method will be described later.

In step S240, the learning unit 104 determines whether or not to end the learning in step S230. For example, the learning unit 104 can determine to end learning when the learning result of the network reaches a predetermined criterion. As an example, the evaluation unit 107 can use the test data (evaluation data) stored in the test data storage unit 106 to evaluate the recognition error rate of the network. This test data can be, for example, a set of learning input data different from the data stored in the basic data storage unit 101 and teacher data indicating the determination result for the learning input data. In addition, the false recognition rate can be defined as the ratio of test data with false recognition results to all the test data used for evaluation. Then, when the recognition error rate of the network is equal to or less than a predetermined threshold, the learning unit 104 can determine that learning is finished. If the learning is not finished, the process returns to step S230, and the learning unit 104 learns the network again. On the other hand, when the learning ends, the process proceeds to step S250, where the learning unit 104 converts the final learning result (for example, the weight parameter of the network and the coupling coefficient of the intermediate stratified output as described later) to the learning result Stored in the storage unit 105 .

The learning device 100 according to this embodiment can be implemented by a device that implements the functional configuration shown in FIG. For example, the learning device 100 may have dedicated hardware that implements each processing unit. On the other hand, part or all of the processing units may be implemented by a computer.

FIG. 11 is a diagram showing the basic configuration of a computer that can operate as the learning device 100 or its processing unit. A processor 1110 in FIG. 11 is, for example, a CPU, and controls the operation of the entire computer. The memory 1120 is, for example, a RAM, and temporarily stores programs, data, and the like. The computer-readable storage medium 1130 is, for example, a hard disk or CD-ROM, and stores programs, data, and the like for a long period of time. In this embodiment, a program that implements the function of each unit stored in the storage medium 1130 is read into the memory 1120 . The processor 1110 operates in accordance with the programs on the memory 1120 to implement the functions of each unit. The memory 1120 or storage medium 1130 can also act as a storage unit such as the basic data storage unit 101 , adaptive data storage unit 103 , learning result storage unit 105 , or test data storage unit 106 .

In FIG. 11, an input interface 1140 is an interface for acquiring information from an external device. An output interface 1150 is an interface for outputting information to an external device. A bus 1160 connects the above units and enables data exchange.

(Hierarchical network configuration and learning method)
An example of a hierarchical network that can be used in this embodiment and the learning of the network performed in step S230 will be described below. FIG. 3 shows an example of a hierarchical network. The network of FIG. 3 is made up of three hidden layer groups 302 , 303 and 304 . Although the specific configuration of each intermediate layer group is not particularly limited, for example, it may be configured by a combination of one or more of a convolutional layer, a pooling layer, and a fully connected layer.

In this embodiment, the hierarchical network provides two or more different outputs corresponding to a single input data. For example, in the network of FIG. 3, outputs are available from two or more different layers. That is, in the network of FIG. 3, hidden layer sub-outputs 307, 308, and 309 are obtained from hidden layer groups 302, 303, and 304, respectively. An integrated output 305 is obtained by integrating the intermediate layer sub-outputs 307 , 308 , and 309 . Based on this integrated output 305, the determination result for the input 301 is obtained. That is, when learning data is input as an input 301, an integrated output 305 is obtained via an intermediate layer group 302, an intermediate layer group 303, and an intermediate layer group 304. FIG. In this embodiment, as an example, hidden layers 302 are made from two convolution layers, hidden layers 303 are made from one pooling layer followed by two convolution layers, and hidden layers 304 are also made from one pooling layer followed by two convolution layers. It shall consist of layers.

In this embodiment, the training of the network is done by side-out learning from each hidden layer. Usually, when using hierarchical networks, error estimation is performed only on the final output, and the network is trained by error backpropagation. On the other hand, in side-out learning, error evaluation is also performed on the output from the hidden layer group. Then, error information can also be input to the hidden layers and backpropagated. For example, HED (Holistically-nested Edge Detection) in Non-Patent Document 1 discloses a method of performing contour extraction (extracting the contour portion of an object included in an input target image) using a hierarchical network. In Non-Patent Document 1, side-out learning is used, and more specifically, error evaluation with learning data is performed even in the intermediate layer, and network learning is performed using error backpropagation.

In the case of this embodiment, hidden layer sub-outputs 307, 308, and 309 are output from respective hidden layer groups 302, 303, and 304 for side-out learning. Then, hidden layer errors 310, 311, and 312, which are errors between the hidden layer sub-outputs 307, 308, and 309 and the learning data (GT), respectively, are calculated. Here, the hidden layer error 310 is denoted as l _side ¹ , the hidden layer error 311 as l _side ² , and the hidden layer error 312 as l _side ³ , respectively. By calculating the sum of the intermediate layer errors 310, 311, and 312 evaluated in this way, an error evaluation value (L _side in Equation (1)) for the entire intermediate layer is obtained.

The error evaluation method is not particularly limited. For example, if the label value of GT is a binary value of 0 and 1, the cross entropy can be used to define the error evaluation value L _side ^m of the hidden layer m as shown in equation (2). In equation (2), y _j ^m represents the output value of each pixel in intermediate layer m. Y ₊ ^m represents a positive region (with a label value of 1) out of the GT given to the intermediate layer m, and Y ₋ ^m represents a negative region (with a label value of 0) out of the GT given to the intermediate layer m. show. Σ means the sum of all pixels. β is a coefficient for correcting an imbalance in the ratio of positive and negative GTs, and can be defined, for example, as the ratio of the number of pixels in the negative region to the number of pixels in the entire GT. This value β may be calculated and set for each GT, or the same value (for example, the average value of the values β for each GT) may be set for all GTs.

Alternatively, integrated output 305 can be obtained by integrating two or more different outputs corresponding to the input data. For example, the hidden layer sub-outputs 307, 308, 309 can be overlaid by taking a linear sum of the hidden layer sub-outputs 307, 308, 309. FIG. Then, an integrated output 305 can be obtained by applying an activation function σ such as a sigmoid function to the linear sum thus obtained. Here, the hidden level output 307 can be represented as A _side ¹ , the hidden level output 308 as A _side ² , and the hidden level output 309 as A _side ³ , respectively. In this case, for example, Y _fuse according to equation (3) can be obtained as integrated output 305 . The weights of the intermediate layer sub-outputs 307, 308, and 309 used to obtain the integrated output 305 can also be determined by learning. For example, the linear sum coupling coefficient h _m shown in equation (3) can also be determined by learning.

In this embodiment, the integrated error 313, which is the error between the integrated output 305 and the GT, is also evaluated. For example, L _fuse , which is the error between the integrated output Y _fuse and the GT label value Y, can be obtained as the integrated error 313 according to equation (4). In Equation 4, Dist() means a distance function used for error evaluation between Y and Y _fuse , and cross entropy, for example, can be used as this function.

The overall network error can be obtained according to the integration error 313 (L _fuse ) and the sum of each hidden layer error 310, 311, 312 (L _side ). For example, the overall network error can be L _total given in equation (5). Each weight parameter in the hierarchical network and the coupling coefficient (h _m ) of the above intermediate layer sub-outputs can be determined by learning so as to minimize the error (L _total ) of the entire network.

The configuration of the hierarchical network as described above and the learning method are as described in Non-Patent Document 1, for example. On the other hand, in this embodiment, when obtaining the hidden layer errors 310, 311, and 312, it is set according to the respective hidden layer groups 302, 303, and 304 (or the hidden layer sub-outputs 307, 308, and 309), Adaptive learning data is used. That is, the hidden layer errors 310, 311, and 312 are the adaptive GTs 306-1, 306-2, and 306-3 set in accordance with the respective hidden layer groups 302, 303, and 304, and the hidden layer standard outputs 307, 308 , 309 and . This configuration will be described below.

FIG. 4 is a diagram for explaining side-out learning of a network when applying a hierarchical network to contour extraction from an image, for example. FIG. 4 schematically shows a case where the same GT (corresponding to basic learning data) is used as in Non-Patent Document 1 to evaluate the error of each intermediate layer sub-output. FIG. 4 shows the relationship between the GT 306 and the integrated output 305 and the hidden sub-outputs 307-309 from the hidden layers 302-304.

In hierarchical networks such as convolutional neural networks, pooling layers are usually placed after the convolutional layers. By arranging the pooling layer, the position sensitivity of the features extracted by the convolutional layer is reduced, and the output from the pooling layer can obtain robustness against changes in position.

For example, if 2×2 MAX pooling with stride 2 is performed in the pooling layer, the pooling will output only the maximum value among the 2×2 4 pixels. As described above, in the example of FIG. 3, the intermediate layer groups 303 and 304 each have one pooling layer. Therefore, for example, when an image that is training input data of 128×128 size is input to the network and these pooling layers perform 2×2 MAX pooling with stride 2, the intermediate layer group 303 outputs 64×64 size. can get. An output of 32×32 size is obtained from the intermediate layer group 304 .

On the other hand, GT (corresponding to basic learning data) is usually an image (for example, a contour image) of the same size as the learning input data. Therefore, in order to compare the hidden layer reference output with GT for error evaluation, the hidden layer reference output is enlarged to 128×128 size, which is the same size as GT. Then, as shown in FIG. 4, in the stage of error evaluation, one pixel in the intermediate layer reference output has a size of 2×2 in the case of the intermediate layer reference output 308, and is reduced to 2×2 in the case of the intermediate layer reference output 309. Enlarged to 4x4 size. Therefore, in the case of contour extraction, for example, even if the contour line width in the intermediate layer output 307 and GT is 1 pixel size, the line width in the intermediate layer output 308 is 2 pixel size, and in the intermediate layer output 309 has a width of 4 pixels. Therefore, when estimating the error, there is a possibility of overestimation of the error due to the difference in line width in the intermediate layer sub-outputs 308 and 309 .

FIG. 5 schematically shows the process of overestimating the error due to the difference in line width between the GT and the intermediate layer output. As shown in FIG. 5A, since there is no line width difference between the intermediate layer output 307 and the GT 306, the difference between the outline pattern output from the intermediate layer group 302 and the pattern of the GT 306 is evaluated in the error evaluation. be done. On the other hand, as shown in FIG. 5B, since there is a difference in line width between the intermediate layer output 308 and the GT 306, in the error evaluation, the outline pattern output from the intermediate layer group 303 and the GT 306 Errors due to line width differences are also evaluated, as well as differences from the pattern of . Furthermore, as shown in FIG. 5C, there is a larger linewidth difference between the intermediate layer output 309 and the GT 306, so the error due to the linewidth difference is larger.

FIG. 5D schematically shows how the error is overestimated. In this way, when there is a difference in line width between the contour pattern 510 output from the intermediate layer group shown in the intermediate layer reference output and the GT 520, correct There is a region 530 where no error evaluation is made. What we want to evaluate correctly in the contour extraction problem is the pattern difference between the output and the GT. occurs.

Non-Patent Document 1 does not describe a process for generating learning data suitable for learning based on errors in intermediate layer standard outputs from basic learning data. Then, when performing side-out learning based on each hidden layer reference output using the same training data (equivalent to basic learning data) based on the error of the final integrated output, the error evaluation of the hidden layer reference output Performance could be degraded and learning less efficient.

Therefore, in this embodiment, the setting unit 102 sets teacher data (adaptive GT) for each of two or more different outputs from the network, corresponding to a single input data for learning. For example, the setting unit 102 can set an adaptive GT for each hidden layer group (or hidden layer reference output). With such a configuration, it is possible to reduce other influences such as line width and more accurately evaluate errors that are originally intended to be evaluated. As a result, it is possible to improve the convergence of side-out learning and the performance of the resulting hierarchical network.

For this reason, the setting unit 102 can set adaptive learning data obtained by processing the original basic learning data for each intermediate layer group. For example, for each hidden layer group, the setting unit 102 is set so that the line width in the hidden layer standard output and the line width of the adaptive GT used for error evaluation are close to each other, or at least so that the error evaluation is not performed excessively. , can generate adaptive training data. In this way, the setting unit 102 can generate learning data so that appropriate error evaluation is performed for each intermediate stratified output.

On the other hand, the basic data storage unit 101 may store learning data (adaptive teacher data) for each of two or more different outputs from the hierarchical network corresponding to single learning input data. . In this case, setting section 102 may acquire adaptive learning data from basic data storage section 101 and store it in adaptive data storage section 103 .

(Method of setting adaptive learning data)
A specific example of a method for setting adaptive learning data in step S220 will be described below.

FIG. 6 is a diagram for explaining a method of setting adaptive learning data according to the present embodiment in the case of using the hierarchical network of FIG. FIG. 6A shows the contour pattern shown in the intermediate layer reference output 307 and the positive area (representing the contour pattern, hereinafter simply referred to as GT) indicated by the GT 601 for error evaluation of the intermediate layer reference output 307. and indicate. Similarly, FIGS. 6(B) and 6(C) show the contour patterns shown in the intermediate layer sub-outputs 308 and 309 and the contour patterns shown in the GTs 602 and 603 for error evaluation of the intermediate layer sub-outputs 308 and 309. and indicate. As already explained, the hidden layer standard outputs 308 and 309 are expanded to match the GT so that the resolution of the hidden layer standard outputs 308 and 309 matches the resolution of the GT. Correspondingly, the line width of the contour pattern shown in the intermediate layer sub-outputs 308 and 309 also increases.

Therefore, the setting unit 102 can set teacher data for two or more different outputs based on the resolutions of the two or more different outputs. For example, the setting unit 102 can set the GTs 601-603 for the intermediate layer output 307-309 based on the resolution of the intermediate layer output 307-309. In this embodiment, the setting unit 102 sets a line drawing pattern having a width corresponding to each of two or more different outputs as teacher data for two or more different outputs. For example, the setting unit 102 can set GTs 601 to 603 representing line drawing patterns having widths corresponding to the resolutions of the intermediate layer standard outputs 307 to 309 for evaluation of the intermediate layer standard outputs 307 to 309 .

Specifically, the line widths of the GTs 602 and 603 for the intermediate layer reference outputs 308 and 309 are increased so that the line widths of the contour-representing line drawing patterns shown in the intermediate layer reference outputs and GT are close to each other. More specifically, in the example of FIG. 6, the line widths of the contour patterns shown in GTs 601, 602, 603 for intermediate layer sub-outputs 307, 308, 309 are 1, 2, 4, respectively. In this way, the setting unit 102 sets the line width of the line drawing pattern to be larger when the resolution of the intermediate layer output is low (the number of pixels is small) compared to when the resolution is high (the number of pixels is large). , adaptive GT can be set. For example, the setting unit 102 can set the adaptive GT such that the line width of the line drawing pattern shown in the adaptive GT substantially matches (the resolution of the basic training data/the resolution of the intermediate layered output).

The setting unit 102 can use the basic learning data to generate adaptive learning data for error evaluation of the intermediate layer standard output. In the case of this embodiment, the setting unit 102 can generate adaptive learning data using basic teacher data, which is a line drawing pattern corresponding to learning input data. The setting unit 102 can generate adaptive learning data (GT911-913) for error evaluation of the intermediate stratified outputs 307-309, for example, according to the flowchart of FIG. 9(D).

In step S<b>901 , the setting unit 102 acquires basic learning data (GT 912 ) stored in the basic data storage unit 101 . In step S902, the setting unit 102 generates GT911 and GT913 by filtering the GT912. In step S903, the setting unit 102 stores GT911-GT913 thus obtained in the adaptive data storage unit 103, thereby setting GT911-913 for each of the intermediate layer reference outputs 307-309.

In this example, the setting unit 102 generated the adaptive learning data by filtering the basic learning data. That is, the setting unit 102 causes different adaptive learning data (GT911, 913) can be obtained. The outline pattern shown in the intermediate layer sub-output changes from a fine form reflecting the texture to a rough form as it approaches the final output side. Such changes can be modeled and the shape of the GT can be altered to accommodate such changes by the effect of filters that apply transformations to the base training data. As an example, the setting unit 102 sets the line width of the line drawing pattern to be larger when the resolution of the intermediate layer sub-output is low (the number of pixels is small) compared to when the resolution is high (the number of pixels is large). , you can choose which filters to use.

A specific example of the filter is a bandpass filter that passes only a specific frequency band. FIG. 9A shows GT911 obtained by applying a high-frequency pass filter to GT912. FIG. 9B shows a GT 912 whose outline pattern has a line width of 2, and the GT 912 is used as it is for the intermediate layer sub-output 308 . FIG. 9C shows GT913 obtained by applying a low frequency pass filter to GT912. As can be seen from FIGS. 9A to 9C, GT911 has a narrower contour pattern line width than GT912, and GT913 has a thicker contour pattern line width than GT912. In the frequency-intensity graphs shown in FIGS. 9A to 9C, the gray portions indicate the bands passed by filtering. Note that the filtering process may be omitted or the contour pattern may be erased for a short contour pattern (for example, the maximum length is 10 pixels or less). According to such processing, for example, an effect of suppressing the influence of noise can be expected.

As another example, the basic data storage unit 101 may store vector data representing contour patterns. In this case, the setting unit 102 can generate a GT having a line width corresponding to the intermediate layer group.

Also, the adaptive learning data (GT601-603) for error evaluation of the intermediate layer sub-outputs 307-309 may be stored in the basic data storage unit 101 in advance. Further, setting section 102 may generate GTs 601 to 603 based on data stored in basic data storage section 101 . FIG. 6D is a diagram for explaining an example of a data storage method for generating GTs 601 to 603 in basic data storage unit 101. As shown in FIG. FIG. 6(E) is an enlarged view of the vertical line portion of FIG. 6(D). As shown in FIGS. 6(D) and 6(E), as GT 601 for error evaluation of integrated output 305 and intermediate layer sub-output 307, a contour pattern indicated by "1" is used. More specifically, GT 601 is the area indicated by "1". Further, as the GT 602 for error evaluation of the intermediate layer reference output 308, contour patterns indicated by "1" and "2" are used, and as the GT 603 for error evaluation of the intermediate layer reference output 309, "1" and The contour patterns indicated by "2" and "3" are used. That is, the positive regions of GT602 are the regions represented by "1" and "2", and the positive regions of GT603 are the regions represented by "1", "2" and "3".

In this case, the setting unit 102 uses the data stored in the basic data storage unit 101 to generate and set adaptive learning data (GTs 601 to 603) for error evaluation of the respective intermediate stratified outputs 307 to 309. be able to. In this way, by sequentially increasing the line widths of the contour patterns in the GTs 601 to 603 for error evaluation of the intermediate layer reference outputs 307 to 309, errors caused by factors other than pattern differences are prevented from being overestimated. Side-out learning can be performed more effectively. For example, the GT for error estimation of the output from the first hidden layer is more likely than the GT for error estimation of the output from the second hidden layer that is downstream through the pooling layer from the first hidden layer. However, GT can be set so that the line width of the contour pattern is thickened.

The setting unit 102 sets the GTs obtained by performing further image processing such as blurring on the respective GTs for intermediate layer standard output obtained as described above, as adaptive learning data. good too. For example, FIGS. 8A to 8C show the results of applying Gaussian blur (processing for blurring an image using a Gaussian function) to GTs 601 to 603 shown in FIG. That is, FIG. 8A shows a cross section 801 after applying Gaussian blur to the GT 601 with a line width of 1 (in the width direction of the contour pattern) for use in error evaluation between the integrated output 305 and the intermediate layer sub-output 307. pixel value distribution). Similarly, in FIGS. 8B and 8C, cross sections 802 and 802 after Gaussian blur is applied to GTs 602 and 603 with line widths of 2 and 4 for use in error evaluation of intermediate layer sub-outputs 308 and 309. 803 is shown. The processing applied to each GT 601-603 may be of the same strength or may be of different strengths to match the characteristics of the intermediate strata output.

In this manner, the setting unit 102 can set the line drawing pattern subjected to the blurring process as teacher data for two or more different outputs. There is a possibility that the correct position of the contour pattern indicated by the learning input data and the position of the contour pattern indicated by the GT are slightly deviated due to an input error. Here, by performing blurring processing (for example, Gaussian blur processing) on the GT, the input error centered on the true position (for example, the input error following the Gaussian distribution) is reflected in the GT, and side-out is performed more effectively. can learn.

So far, the configuration in which the line width of the outline pattern in the GT is mainly changed according to the characteristics of the intermediate stratified output has been described, but the adaptive learning data setting method is not limited to such a method. For example, the setting unit 102 can set teacher data for two or more different outputs in which an error evaluation non-target area having a width corresponding to each of the two or more different outputs is set around the line drawing pattern. .

A method of setting an error-evaluation-excluded region in the GT where error evaluation is not performed in this way will be described with reference to FIG. FIG. 7(A) shows the hidden layer sub-output 307 and the GT 601 for error estimation, which is similar to FIG. 6(A). On the other hand, FIG. 7B shows the error evaluation of the intermediate layer sub-output 308, and the error evaluation of the intermediate layer sub-output 308 composed of the GT 601 (positive region of GT) with a line width of 1 and the incidental region 702 with a line width of 2. represents GT. In addition, FIG. 7(C) shows the error evaluation of the intermediate layer secondary output 309 and the intermediate layer secondary output 309 composed of the GT 601 (positive region of GT) with a line width of 1 and the incidental region 703 with a line width of 4. represents GT. Here, the incidental area means an area attached to both sides of the contour pattern, which is a positive area and is not evaluated in the error evaluation. In this case, in the evaluation using equation (2), Y ₊ ^m represents a positive region (for example, the label value is 1) of the GT given to the intermediate layer m. Also, Y ₋ ^m represents a negative region (for example, the label value is 0) of the GT given to the intermediate layer m. This negative area is the area excluding the positive area and the incidental area (for example, the label value is 2) from the entire area.

A GT having such an incidental area can be created, for example, according to the data shown in FIGS. 6(D) and 6(E). For example, the GT shown in FIG. 7B can be created by setting the area of "1" as a positive area and the area of "2" as an incidental area. Also, the GT shown in FIG. 7(C) can be created by setting the "1" area as the positive area and the "2" and "3" areas as the incidental areas. Moreover, it is also possible to set the incidental area using the filtering process as described above. In this way, by sequentially increasing the line widths of the incidental regions 702 and 703 in the GT 601 for error evaluation of the intermediate layer sub-outputs 307 to 309, the overestimation of errors caused by factors other than pattern differences can be prevented. It is possible to prevent side-out learning more effectively.

(Application examples for various network configurations)
Up to this point, a case has been described where side-out learning is performed based on the intermediate layer output from each layer group, but the application example of the method according to the present embodiment is not limited to this. For example, as shown in FIG. 10, side-out learning can also be performed based on multiple outputs from one hidden layer group. In the configuration shown in FIG. 10, side-out learning is performed based on outputs from two or more different hidden layers in one hidden layer group of the network. In FIG. 10A, one hidden layer group 1300 includes convolutional layers 1301, 1302, 1303 and a pooling 1304 layer. FIG. 10A also shows outputs 1311 to 1313 of convolution layers 1301 to 1303 and GTs 1321 to 1323 used for error evaluation there. FIG. 10B shows changes in the line width of contour patterns in GT1321 to GT1323, and it can be seen that the line width gradually increases.

In this case, the setting unit 102 can set teacher data for learning input data for each of outputs from two or more different hidden layers in one hidden layer group of the network. For example, the line widths of contour patterns in GTs 1321 to 1323 for error evaluation of outputs 1311 to 1313 can be made thicker sequentially. As a specific example, the setting unit 102 sets the error of the output from the second hidden layer, which is downstream of the first hidden layer through the convolutional layer, from the GT for error evaluation of the output from the first hidden layer. The GT for evaluation can be set so that the line width of the contour pattern is thicker than the GT for evaluation. With such a configuration, the influence of the expansion of the spatial interdependence range of pixels due to the sequential application of filters in the convolution layers is incorporated, and errors caused by factors other than pattern differences are prevented from being overestimated. be able to. Therefore, side-out learning can be performed more effectively.

As another example, side-out learning can also be performed based on multiple outputs from one hidden layer of the network, as shown in FIG. 9(E). As an example, FIG. 9E shows a case where an intermediate layer group 950 is composed of convolution layers 951 to 953 and a pooling layer 954 . In the example of FIG. 9E, the setting unit 102 can set teacher data for learning input data for each of outputs from two or more different channel groups in one layer of the network.

For example, the setting unit 102 can separate an image represented by basic learning data according to a predetermined condition and generate a plurality of adaptive learning data representing respective partial images. As a specific example, the contour patterns shown in GT may be separated for each specific direction, and the respective contour patterns may be used to learn the weighting coefficients (convolution filters) of the corresponding networks. Here, the convolutional layer 951 that outputs side-out is divided into a convolutional layer 961 and a convolutional layer 962 . Convolutional layer 961 and convolutional layer 962 correspond to different channel groups in convolutional layer 951 . Here, the setting unit 102 can set GTs having different directional components in each of the convolution layers 961 and 962 . In this case, the learning of the weighting factors for each of the convolutional layers 961 and 962 is performed using GTs with different directional components. For example, convolutional layer 961 is trained using GT971, which indicates contour patterns in a first direction, and convolutional layer 962 is trained using GT972, which indicates contour patterns in a second direction different from the first direction. It can be carried out. In this way, by performing intensive training using GTs having specific patterns for each convolutional layer, it is expected that the overall recognition performance will be improved. Such an arrangement can be combined with the various arrangements described above, for example with applying further image processing such as Gaussian blurring to the GT.

So far, the case where GT is set for each intermediate stratum output has been described on the premise that the intermediate stratum output is expanded in accordance with the GT. On the other hand, the setting unit 102 may set the GT according to each size of the intermediate layer standard output. For example, the setting unit 102 may reduce the GT (basic learning data) indicating the contour pattern to match the size of the intermediate layer reference output. A specific example is a method of generating adaptive learning data by performing filtering on basic learning data. For example, if the basic training data is a binary image (the "1" values represent contours), 2×2 MAX pooling with a stride of 2×2 provides a resolution while maintaining the contour pattern shown in the basic training data. is halved, adaptive training data can be obtained. In this way, instead of simply thinning out or repeating pixels to generate adaptive learning data from basic learning data, image processing such as filtering is performed on the basic learning data to generate adaptive learning data. can do. According to such a method, it is possible to generate adaptive learning data suitable for intermediate layer standard output.

By learning the hierarchical network by the method described above, the parameters of the hierarchical network can be created. Further, the information processing apparatus according to one embodiment includes a generation unit that generates a result of recognition processing corresponding to input data using the hierarchical network in which the created parameters are set. Such a hierarchical network can be implemented by a program, or by an arithmetic device having a memory for storing parameters and an arithmetic unit such as a GPU. According to the method of the present embodiment, each of two or more different outputs from the hierarchical network are evaluated using the same adaptive training data, instead of using the same basic training data as in the conventional method. evaluated. Therefore, the parameters of the network obtained by learning are different from the conventional ones and are more suitable for recognition processing of input data.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

100: learning device, 102: setting unit, 104: learning unit

Claims (16)

A learning device for learning a neural network used for attribute determination processing of each pixel of an image ,
setting means for setting teacher data for each of two or more different outputs from the neural network corresponding to a single input data for learning;
Learning means for learning the neural network based on the error between each of the two or more different outputs obtained by inputting the learning input data to the neural network and teacher data corresponding to the output. and
The training data for each of the two or more different outputs indicates the results of different deformation processing or filtering processing on the basic training data corresponding to the single learning input data, and the basic training data is the single learning input data. is image data indicating the attribute of each pixel indicated by the learning input data of . 2. The learning apparatus according to claim 1, wherein said setting means sets teacher data for each of said two or more different outputs based on the structure of said neural network. 3. The learning apparatus according to claim 1, wherein said setting means sets teacher data for learning input data for each of outputs from two or more different hierarchies of said neural network. 3. The setter according to claim 1, wherein said setting means sets teacher data for learning input data for each of outputs from two or more different channel groups in one layer of said neural network. learning device. 5. The learning according to any one of claims 1 to 4, wherein said setting means sets the teacher data for said two or more different outputs based on the resolutions of said two or more different outputs. Device. The setting means sets an error non-evaluation area for teacher data for each of the two or more different outputs based on basic teacher data for outputs from the neural network corresponding to the single input data for learning. 6. The learning device according to any one of claims 1 to 5, characterized in that: 7. The learning device according to any one of claims 1 to 6 , wherein the result of said attribute determination processing indicates a line drawing pattern. A learning device for learning a neural network,
setting means for setting teacher data for each of two or more different outputs from the neural network corresponding to a single input data for learning;
Learning means for learning the neural network based on the error between each of the two or more different outputs obtained by inputting the learning input data to the neural network and teacher data corresponding to the output. and
the neural network after learning provides two or more different outputs corresponding to input data, and a result of integration of the two or more different outputs indicates a result of recognition processing for the input data;
the input data is image data, and the result of recognition processing for the input data is attribute information of each pixel of the image data;
a result of recognition processing for the input data is a line drawing pattern corresponding to the input data;
The learning device, wherein the setting means sets a line drawing pattern having a width corresponding to each of the two or more different outputs as teacher data for the two or more different outputs. A learning device for learning a neural network,
setting means for setting teacher data for each of two or more different outputs from the neural network corresponding to a single input data for learning;
Learning means for learning the neural network based on the error between each of the two or more different outputs obtained by inputting the learning input data to the neural network and teacher data corresponding to the output. and
the neural network after learning provides two or more different outputs corresponding to input data, and a result of integration of the two or more different outputs indicates a result of recognition processing for the input data;
the input data is image data, and the result of recognition processing for the input data is attribute information of each pixel of the image data;
a result of recognition processing for the input data is a line drawing pattern corresponding to the input data;
The learning device, wherein the setting means sets a line drawing pattern subjected to blurring processing as teacher data for the two or more different outputs. A learning device for learning a neural network,
setting means for setting teacher data for each of two or more different outputs from the neural network corresponding to a single input data for learning;
Learning means for learning the neural network based on the error between each of the two or more different outputs obtained by inputting the learning input data to the neural network and teacher data corresponding to the output. and
the neural network after learning provides two or more different outputs corresponding to input data, and a result of integration of the two or more different outputs indicates a result of recognition processing for the input data;
the input data is image data, and the result of recognition processing for the input data is attribute information of each pixel of the image data;
a result of recognition processing for the input data is a line drawing pattern corresponding to the input data;
The setting means sets teacher data for the two or more different outputs, in which an error evaluation non-object area having a width corresponding to each of the two or more different outputs is set around the line drawing pattern. and learning device. 11. The learning device according to any one of claims 7 to 10, wherein said setting means generates said teacher data using basic teacher data that is a line drawing pattern corresponding to said input data for learning. . A learning device for learning a neural network,
setting means for setting teacher data for each of two or more different outputs from the neural network corresponding to a single input data for learning;
Learning means for learning the neural network based on the error between each of the two or more different outputs obtained by inputting the learning input data to the neural network and teacher data corresponding to the output. and
the neural network after learning provides two or more different outputs corresponding to input data, and a result of integration of the two or more different outputs indicates a result of recognition processing for the input data;
the input data is image data, and the result of recognition processing for the input data is attribute information of each pixel of the image data;
a result of recognition processing for the input data is a line drawing pattern corresponding to the input data;
The learning device, wherein the setting means generates the teacher data by filtering basic teacher data, which is a line drawing pattern corresponding to the input data for learning . A method for creating a trained neural network used for attribute determination processing of each pixel of an image ,
A setting step of setting teacher data for each of two or more different outputs from the neural network corresponding to a single learning input data;
A learning step of learning the neural network based on an error between each of the two or more different outputs obtained by inputting the learning input data to the neural network and teacher data corresponding to the output. and
The training data for each of the two or more different outputs indicates the results of different deformation processing or filtering processing on the basic training data corresponding to the single learning input data, and the basic training data is the single learning input data. is image data indicating the attribute of each pixel indicated by the learning input data . A neural network for causing an information processing device to generate a processing result of recognition processing corresponding to input data, wherein the neural network is set with parameters created by the creating method according to claim 13. 15. An information processing apparatus comprising processing means for generating a processing result of recognition processing corresponding to input data using the neural network according to claim 14. A program for causing a computer to function as each means of the learning device according to any one of claims 1 to 12.

Images