JP2023086549A - Information processing apparatus, information processing method in information processing apparatus, and program - Google Patents

Abstract

To provide an information processing apparatus configured to perform analysis or recognition with high accuracy while suppressing increase of data volume or processing load in artificial intelligence using CNN.SOLUTION: An information processing apparatus 1A includes: a CNN 114 which includes a convolutional neural network including a convolution layer and executes convolution processing on data having multiple channels; a first transformer 112 which performs non-linear transformation on data input to the information processing apparatus 1A to be input to the CNN 114; and/or an inverse transformer 115 which performs non-linear transformation on data output from the CNN 114 to be output from the information processing apparatus 1A. The first transformer 112 and/or the CNN 114 performs non-linear transformation on data separately by channel.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing apparatus and information processing method for processing data using a convolutional neural network (CNN).

In recent years, convolutional neural networks (CNN, hereinafter referred to as "CNN") are often used to analyze and recognize data using artificial intelligence (AI). For example, CNNs are often used in various types of analysis and recognition of image data, voice data, and the like. Conventionally, as an artificial intelligence system using such a CNN, in order to improve the accuracy of analysis and recognition by the CNN, data with multiple parameters as discrete values, for example, digital color image data in the RGB color space are nonlinearly An invention is known in which a converter that performs space conversion is provided in the front stage of a CNN (see, for example, Patent Document 1).

Japanese Patent No. 6476531

However, the purposes of CNN are diverse, such as data recognition, data analysis, and improvement of data accuracy. Depending on the type and purpose of the data, the effect of CNN processing may be enhanced by nonlinearly transforming only a specific parameter out of a plurality of parameters. However, in Patent Document 1, since all of the parameters of the data to be converted are non-linearly converted, there is a problem that the processing load becomes excessive and the processing accuracy decreases.

The present invention has been made in view of such problems, and in artificial intelligence using CNN, information that can be analyzed and recognized with high accuracy while preventing the amount of data and processing load from becoming excessive. An object of the present invention is to provide a processing device, an information processing method, and a program.

In order to solve such a problem, the invention according to claim 1 is an information processing device comprising a convolutional neural network including a convolutional layer and comprising data processing means for performing convolution processing on data having a plurality of channels, conversion means for performing nonlinear conversion on data input to the information processing device and inputting the data to the data processing means, and/or performing nonlinear conversion on data output from the data processing means First nonlinear processing means comprising inverse transforming means for outputting from the information processing device, wherein the transforming means and/or the inverse transforming means perform the nonlinear transform on the data separately for each of the channels. characterized by comprising

The invention according to claim 2 is, in addition to the configuration according to claim 1, wherein the conversion means and/or the inverse conversion means includes a processing layer group consisting of at least three processing layers, and the processing layer The group consists of an input layer with one node, an intermediate processing layer that is a convolutional layer or dense layer with a plurality of nodes provided after the input layer, and the number of nodes provided after the intermediate processing layer. includes an output layer which is one or more convolutional layers or dense layers, and a processing layer group is provided for each channel of the data input to the convolutional neural network.

The invention according to claim 3 is characterized in that, in addition to the configuration according to claim 2, the intermediate treatment layer is composed of one layer.

The invention according to claim 4 is characterized in that, in addition to the configuration according to claim 2, the intermediate treatment layer is composed of a plurality of layers.

The invention according to claim 5 is, in addition to the configuration according to any one of claims 1 to 4, wherein the transforming means and/or the inverse transforming means combine a plurality of the channels to obtain the nonlinear It is characterized by comprising a second non-linear processing means for performing conversion of .

The invention according to claim 6, in addition to the configuration according to any one of claims 1 to 5, is a storage storing a conversion table in which conversion modes used in the first nonlinear processing means are recorded. wherein the first nonlinear processing means performs the nonlinear conversion using the conversion table acquired from the storage means.

The invention according to claim 7 is characterized in that, in addition to the configuration according to any one of claims 1 to 6, a skip connection is used in the transforming means and/or the inverse transforming means.

According to an eighth aspect of the present invention, there is provided an information processing method in an information processing apparatus, comprising a data processing procedure in which convolution processing is performed on data having a plurality of channels in a convolutional neural network including convolution layers, wherein A transformation procedure that performs nonlinear transformation on data input to an information processing device and is input to the processing of the data processing procedure, and / or a nonlinear transformation of data that is output by the processing of the data processing procedure. An inverse transformation procedure is provided for performing transformation and outputting it from the information processing device, wherein the transformation procedure and/or the inverse transformation procedure performs the nonlinear transformation separately on the data for each channel. It is characterized by having one nonlinear processing procedure.

According to a ninth aspect of the present invention, there is provided a program that causes a computer to function as the information processing apparatus according to any one of the first to seventh aspects.

According to the present invention, in artificial intelligence using CNN, it is possible to perform analysis and recognition with high accuracy while preventing the amount of data and processing load from becoming excessive.

1 is a functional block diagram showing the overall configuration of an information processing apparatus according to Embodiment 1; FIG. 3 is a functional block diagram schematically showing the detailed configuration of an image processing unit of the information processing apparatus; FIG. 3 is a functional block diagram schematically showing the detailed configuration of an image processing unit of the information processing apparatus; FIG. It is a functional block diagram which shows the detailed structure of the 1st converter of an information processing apparatus same as the above. It is a functional block diagram which shows the outline of the modification of the 1st converter of the information processing apparatus same as the above. It is a functional block diagram which shows the detailed structure of the 2nd converter of an information processing apparatus same as the above. It is a block diagram and a time chart which show typically the structure and processing procedure (data processing procedure) of CNN of an information processing apparatus same as the above. FIG. 11 is a functional block diagram showing the configuration of a first converter of the information processing device of this embodiment 2; FIG. 12 is a functional block diagram showing a part of the configuration of an image processing unit of the information processing apparatus according to the third embodiment; FIG. 14 is a functional block diagram showing a part of the configuration of an image processing section of the information processing apparatus according to the fourth embodiment; FIG. 12 is a functional block diagram showing a part of the configuration of an image processing unit of the information processing apparatus according to the fifth embodiment; FIG. 12 is a functional block diagram showing a part of the configuration of an image processing section of the information processing apparatus according to Embodiment 6; FIG. 14 is a functional block diagram showing a part of the configuration of an image processing section of the information processing apparatus according to Embodiment 7; 1 is a functional block diagram showing a part of the configuration of an image processing unit of an information processing apparatus as conventional example 1, and (B) an image processing unit of an information processing apparatus as conventional example 2, as embodiments of the present invention. FIG. 4C is a functional block diagram showing a part of the configuration, (C) a functional block diagram showing a part of the configuration of the image processing unit of the information processing apparatus as the present invention;

[Embodiment 1 of the invention]
1 to 7 show an information processing apparatus and an information processing method in the information processing apparatus according to the first embodiment. Embodiment 1 of the present invention will be described below with reference to the drawings.

[Basic configuration]
First, the configuration of the information processing apparatus according to the first embodiment will be described.

The information processing device 1A of the first embodiment shown in FIG. Restore the data that was saved. The information processing device 1A performs data processing using CNN on digital data.

In the following description of the first embodiment, the information processing apparatus 1A analyzes, recognizes, and restores image data as digital data. Further, the image data input to the information processing apparatus 1A of the first embodiment is 256-tone RGB color model image data (image data having three parameters of R value, G value, and B value). shall be

However, data handled by the information processing apparatus 1A is not limited to image data, and may be voice data as digital data or various digital data other than voice. Further, the data handled by the information processing apparatus 1A may be converted from analog data into digital data and subjected to various processes.

The image data handled in the first embodiment may be image data other than the RGB color model, for example, image data obtained by converting the RGB color model into a different color space such as YUV or YCbCr. Image data having parameters (for example, image data having four parameters of RGBY) may be used. In this case, the functional means of the information processing apparatus 1A described below are configured according to the types of parameters and the number of parameters.

[Functional Means of Information Processing Device]
As shown in FIG. 1, the information processing apparatus 1A of the first embodiment includes a control unit 10, an image processing unit 11, a storage unit 12 as a "storage unit", a communication unit 13, a display unit 14, and an operation unit as functional units. A part 15 is provided. The operation of the information processing apparatus 1A will be described below as one server computer, but it may be configured such that processing is distributed among a plurality of computers.

The control unit 10 uses a processor such as a CPU (Central Processing Unit), a memory, and the like, and controls components of the device to realize various functions. The image processing unit 11 uses a processor such as a GPU (Graphics Processing Unit) or a dedicated circuit and a memory, and executes image processing according to control instructions from the control unit 10 . Note that the control unit 10 and the image processing unit 11 are configured as one piece of hardware (SoC: System on a Chip) in which a processor such as a CPU or GPU, a memory, a storage unit 12 and a communication unit 13 are integrated. good too.

The storage unit 12 is various storage media such as a hard disk or flash memory. The storage unit 12 stores an image processing program 1P, a CNN library 1L for DL (Deep Learning), particularly a CNN library 1L, and a converter library 2L. In addition, in the storage unit 12, information defining the CNN 114, the first transformer 112, the second transformer 113, the inverse transformer 115, and the weight of each layer in the learned CNN 114, which are created for each learning Parameter information and the like including coefficients and the like are stored.

A conversion table 121 is stored in the storage unit 12 . This conversion table 121 is read into the first converter 112 and used for arithmetic processing in the first converter 112 (detailed in [Conversion table] described later).
The communication unit 13 is a communication module that realizes communication connection to a communication network such as the Internet. The communication unit 13 uses a network card, a wireless communication device, or a carrier communication module.

The display unit 14 uses a liquid crystal panel, an organic EL (Electro Luminescence) display, or the like. The display unit 14 can display an image by processing in the image processing unit 11 according to instructions from the control unit 10 .

The operating unit 15 includes a user interface such as a keyboard or mouse. A physical button provided on the housing may be used. Also, software buttons or the like displayed on the display unit 14 may be used. The operation unit 15 notifies the control unit 10 of operation information by the user.

The reading unit 16 uses a disk drive, for example, and can read the image processing program 2P, the CNN library 3L, and the converter library 4L stored in the recording medium 2 using an optical disk or the like. The image processing program 1P, the CNN library 1L, and the converter library 2L stored in the storage unit 12 control the image processing program 2P, the CNN library 3L, and the converter library 4L read by the reading unit 16 from the recording medium 2. It may be one that the unit 10 duplicates in the storage unit 12 .

The control unit 10 of the information processing apparatus 1A functions as an image processing execution unit 101 as a "learning execution unit" based on the image processing program 1P stored in the storage unit 12. FIG. The image processing unit 11 functions as a CNN 114 (CNN engine) using a memory based on the CNN library 1L, definition data, and parameter information stored in the storage unit 12, and also functions as a CNN 114 (CNN engine) based on the converter library 2L and filter information. to function as the first converter 112 and the second converter 113 . The image processing unit 11 may function as an inverse transformer 115 depending on the types of the first transformer 112 and the second transformer 113 .

[Functional Means of Image Processing Execution Unit]
As shown in FIG. 2, the image processing execution unit 101 includes, as functional means, an input unit 111, a first converter 112 as a "conversion means" and a "first nonlinear processing means", a "conversion means" and a "second A second transformer 113 as a "nonlinear processing means", a CNN 114 as a "data processing means", an inverse transformer 115 as an "inverse transforming means", and an output unit 116 are provided. The image processing execution unit 101 uses these functional means to give data to each of them and to acquire the data output from each of them.

Specifically, the image processing execution unit 101 inputs image data, which is input data input to the input unit 111 based on the user's operation using the operation unit 15, to the first converter 112, The image data output from the first converter 112 is input to the second converter 113 . The image processing execution unit 101 inputs the data output from the second converter 113 to the CNN 114 . The image processing execution unit 101 inputs the data output from the CNN 114 to the inverse transformer 115 as necessary, inputs the data output from the inverse transformer 115 to the output unit 116, and outputs the input data to the output unit. 116 as output data and input to the storage unit 12 . The image processing execution unit 101 may give the output data to the image processing unit 11 to render it as an image and output it to the display unit 14 .

The CNN 114 has multiple stages of convolution layers and pooling layers defined by definition data, and a fully connected layer (see FIG. 7), extracts the feature amount of the input data, and performs classification based on the extracted feature amount. (detailed in [CNN Configuration and Processing Procedure] below).

The first converter 112 and the second converter 113 each include a convolutional layer and a multi-channel layer like the CNN 114, and perform nonlinear conversion on input data. Here, the non-linear transformation refers to processing such as color space transformation and level correction that non-linearly distorts input values. The inverse transformer 115 includes a convolutional layer and a multi-channel layer for inverse transformation. The inverse transformer 115 functions to restore the distortion caused by the first transformer 112 as the "second nonlinear processing means" and the second transformer 113 as the "first nonlinear processing means". However, the conversion by the inverse converter 115 is not limited to the conversion that is symmetrical with the first converter 112 and the second converter 113 .

[First converter]
3 and 4 schematically show the configuration of the first converter 112 of the first embodiment.

A first transformer 112 performs a non-linear transform on the data separately for each channel. Here, the channel means the R value, G value, and B value (color channel) in the image data of the color image of the RGB color model. That is, this image data is 3-channel data.

As shown in FIG. 4, the first converter 112 includes an R converter 112r, a G converter 112g, and a B converter 112b. The R converter 112r has a first layer (input layer) 112r1 having one node, and a second layer (CONV) having a plurality of nodes and a dense layer formed by the plurality of nodes. It is composed of an (intermediate processing layer) 112r2 and a third layer (output layer) 112r3 having one node. The G converter 112g and the B converter 112b also have the same configuration as the R converter 112r. That is, the G converter 112g has a first layer 112g1, a second layer 112g2 and a third layer 112g3, and the B converter 112b has a first layer 112b1, a second layer 112b2 and a third layer 112b3.

As shown in FIGS. 3 and 4, the second layer 112r2 of the R converter 112r constituting the second layer, which is the intermediate processing layer, has, for example, ₂₅₆ nodes 1120 ₀₀₁ , 1120 _{002 ,} . Prepare. Since the number of nodes is proportional to the processing accuracy, the greater the number of nodes, the higher the processing accuracy. As shown in FIG. 3, the G converter 112g and the B converter 112b are similarly provided with 256 nodes 1120 ₀₀₁ , 1120 ₀₀₂ , . . . 1120 ₂₅₆ , respectively.

The first converter 112 has the function of performing nonlinear transformation on the input and performing processing that nonlinearly distorts the input sample values (transformation procedure, first nonlinear processing procedure). The second layers 112r2, 112g2, and 112b2 of the R converter 112r, the G converter 112g, and the B converter 112b of the first converter 112 are not limited to being configured as dense layers, but are configured as convolution layers. can be anything.

[Specific Configuration of First Converter]
FIG. 4 is a functional block diagram showing a specific configuration of the first converter 112 of this first embodiment.

The R converter 112r of the first converter 112 has a first layer node 112r1 which is an input layer, a second layer 112r2 which is an intermediate processing layer, and a third layer 112r3 which is an output layer. 1121 ₂₅₅ , _{1121 256} as 256 nodes 1121 ₀₀₁ , 1121 ₀₀₂ , . , 1122 ₀₀₂ , . . . 1122 ₂₅₅ , 1122 ₂₅₆ are obtained. The third layer 112r3, which is the output layer of the R converter 112r of the first converter 112, comprises a convolution node _{112r3_1} and an output node _{112r3_2} . The convolution node 112r3 ₁ convolves the outputs processed by the elu activation function at the nodes _{1122 001 ,} 1122 ₀₀₂ , _. In addition, elu activation function processing is performed on the result of convolution. Output node _{112r3_2} outputs the result of the processing in convolution node _{112r3_1} .

This elu (exponential linear unit) is one of the activation functions, and by using elu, data can be transformed nonlinearly. The reason why the first converter 112 uses elu as an activation function is that the curve of the input data (RGB The deformation of the characteristic curve, etc., with parameters such as the numerical value and the brightness of can be done.)

Although not shown in FIGS. 3 and 4, the G converter 112g and the B converter 112b of the first converter 112 have the same configuration as the R converter 112r.

1122 ₂₅₅ , 1122 _{256 of} the second layer 112r2 and the elu activation function of _the third layer. At least one of the processing units _112r32 may not be provided, and any function other than the elu activation function may be used. This is the same for the G converter 112g, the B converter 112b, the second converter 113, and the first inverse converter 115a and the second inverse converter 115b of the inverse converter 115 of the first converter 112. is.

Note that the R converter 112r, G converter 112g, and B converter 112b shown in FIG. 3 and FIG. It is not limited to this and may be decreased or increased. This is the same for the second converter 113, the first inverse transform unit 115a of the inverse transform unit 115, the R inverse transform unit 115br, the G inverse transform unit 115bg, and the B inverse transform unit 115bb of the second inverse transform unit 115b. is.

[Modification of Configuration of First Converter]
FIG. 5 is a functional block diagram outlining a modification of the configuration of the first converter 112 of the first embodiment.

This figure shows an outline of a modification of the R converter 112r of the first converter 112. FIG. In FIG. 5, the first transformer 112 comprises a convolution node _112r34 , a skip connection _112r35 and an activation function processing node _112r36 in the third layer 112r3. The skip connection _112r33 and the convolution node _112r34 convolve the output of the second layer 112r2 with a 1×1 filter. The skip connection _112r33 inputs the data output from the first layer 112r1 to the third layer 112r3 without performing the processing of the second layer 112r2. The activation function processing node _{112r3_6} adds the data processed by the convolution node _{112r3_4} and the data supplied from the skip connection _{112r3_3} , and performs the elu activation function processing of the added data. By providing the skip connection _112r33 , it is possible to appropriately avoid the data gradient vanishing problem that may occur in machine learning.

Although not shown, the G converter 112g and the B converter 112b are also provided with similar skip connections to obtain the same effect. This is the same for the first converter 112 of [Embodiment 2 of the invention] to [Embodiment 8 of the invention] described later.

[Second converter]
3 and 6 schematically show the configuration of the second converter 113 of the first embodiment.

The second converter 113 includes first layers 1131r, 1131g, and 1131b having a plurality of nodes, for example, three, and second layers 1132 ₀₀₁ and 1132 ₀₀₂ that perform 1×1 filter convolution (CONV) as intermediate processing layers. , 1132 ₂₅₅ , 1132 ₂₅₆ and third layers 1133 ₁ , 1133 ₂ , 1133 ₃ that obtain three-channel outputs by convolution of 1×1 filters.

In this Embodiment 1, the number of nodes of the first layers 1131r, 1131g, 1131b and the third layers 1133 ₁ , 1133 ₂ , 1133 ₃ of the second converter 113 is 3. R It is the same number as the converters 112r, G converters 112g, and B converters 112b. That is, the number of nodes in the first layers 1131r, 1131g, and 1131b and the third layers 1133 ₁ , 1133 ₂ , and 1133 ₃ of the second converter 113 is three types of R, G, and B, which are color information of the RGB color model. corresponds to this. ).

Note that the number of nodes of the first layers 1131r, 1131g, 1131b and the third layers 1133 ₁ , 1133 ₂ , 1133 ₃ of the second converter 113 and the number of nodes of the converters 112r, 112g, 112b does not necessarily have to match. Further, in the first embodiment, the first layers 1131r, 1131g, 1131b and the third layers 1133 ₁ , 1133 ₂ , 1133 ₃ of the second converter 113 have the same number of nodes, but they have different numbers of nodes. may Furthermore, the second converter 113 _{is not} limited to the second layers 1132 ₀₀₁ , 1132 ₀₀₂ , .

[Inverse converter]
FIG. 3 schematically shows the configuration of the inverse transformer 115 of the first embodiment.

The inverse transformer 115 includes a first inverse transform section 115a and a second inverse transform section 115b as "first nonlinear processing means".

The first inverse transformation unit 115a has the same configuration as the second transformer 113, and performs inverse transformation of the transformation by the second transformer 113 (inverse transformation procedure). Specifically, the first inverse transform unit 115a uses first layers 115a1 ₁ , 115a1 ₂ , and 115a1 ₃ having a plurality of nodes, for example, three, and a dense layer (DENSE) having a larger number of nodes than the first layer. 115a2 ₀₀₂ , _. . _. 115a2 ₀₀₂ , _{115a2 002 , .} The third layers 115a3 ₁ , 115a3 ₂ and 115a3 ₃ having the same number of nodes as the layers 115a1 ₁ , 115a1 ₂ and 115a1 ₃ are formed.

The second inverse transformation unit 115b has the same configuration as the first transformer 112, and performs inverse transformation of the transformation by the first transformer 112 (inverse transformation procedure). The second inverse transform unit 115b performs nonlinear transform on the data separately for each channel. Channels here refer to R, G, and B values in the image data of the color image of the RGB color model, as in the case of the first converter 112 .

Specifically, the second inverse transforming unit 115b includes an R inverse transforming unit 115br corresponding to the R converter 112r, a G inverse transforming unit 115bg corresponding to the G converter 112g, and a B inverse transforming unit corresponding to the B converter 112b. A portion 115bb is provided. The R inverse transform unit 115br includes _a first layer 115br1 having one node, and second layers 115br2 ₀₀₁ , 115br2 ₀₀₂ , . , and a third layer 115br3 having one node. 115bg2 ₀₀₁ , 115bg2 ₀₀₂ , _. It is a configuration with

Like the second converter 113, the first inverse transform unit 115a performs nonlinear transform on the input to nonlinearly distort the input sample values. The R inverse transforming unit 115br, the G inverse transforming unit 115bg, and the B inverse transforming unit 115bb of the second inverse transforming unit 115b are similar to the R transforming unit 112r, the G transforming unit 112g, and the B transforming unit 112b of the first transforming unit 112. Secondly, it has the effect of performing nonlinear transformation on the input and performing processing that nonlinearly distorts the input sample values (first nonlinear processing procedure).

Note that the first inverse transforming unit 115a, like the second transforming unit 113, nonlinearly transforms the input and performs processing to nonlinearly distort the input sample values. The R inverse transforming unit 115br, the G inverse transforming unit 115bg, and the B inverse transforming unit 115bb of the second inverse transforming unit 115b also perform nonlinear transformation on the input, and perform processing to nonlinearly distort the input sample values.

Further, as described in [Functional Means of Image Processing Execution Unit] above, the processing of the first inverse transforming unit 115a may not be the completely reverse processing of the second transforming unit 113, and the second inverse transforming The processing of the part 115b may not be the completely reverse processing of the first converter 112.

Further, when the output data of machine learning by the information processing apparatus 1A is in the same format as the input data (for example, when image data is output in response to the input of image data), the inverse transformer 115 is more suitable for processing. can do On the other hand, for example, when the output data from the information processing apparatus 1A is in a format different from the input data (for example, when the image recognition result is output as data such as characters and symbols for the input of image data), the inverse converter 115 is often unnecessary. Therefore, the inverse transformer 115 of the first embodiment may not be included in the information processing apparatus 1A depending on the type of data processed by the information processing apparatus 1A and the output mode of the processing result (described later). [Embodiment 4, 5, 7], etc.).

[Conversion table]
The R converter 112r, the G converter 112g, and the B converter 112b, which constitute the first converter 112 of the first embodiment, each use the conversion table 121 in arithmetic processing. As shown in FIG. 2, this conversion table 121 is stored in the storage unit 12, and the first converter 112 fetches it from the storage unit 12 and uses it for calculation.

Specifically, in the conversion table 121, each of the converters 112r _, 112b _, and 112g has 256 kinds of operations, which are the number of nodes in the second layer 1120 ₀₀₁ , 1120 ₀₀₂ , . patterns are recorded. Each converter 112r, 112b, 112g uses this conversion table 121 to perform processing corresponding to actual calculation.

Such processing using the conversion table 121 is possible because the types of operations of the R converter 112r, the G converter 112g, and the B converter 112b in the configuration of the first embodiment are practically as many as the number of nodes. This is because the number of calculation patterns is small and the calculation patterns can be easily recorded as the conversion table 121 .

The first converter 112 and the second converter 113 require a convolution operation (binary operation). In the second converter 113, there are many variations in the values that are input to the nodes of the second layer, and it is difficult to create a table that covers all of these variations. On the other hand, the R converter 112r, the G converter 112g, and the B converter 112b that form the first converter 112, and the R inverse converter 115br and the G inverse converter 115bg that form the second inverse converter 115b. _{, 1120 002 ,} . is. Therefore, variations of each node in the second layer 1120 ₀₀₁ , 1120 ₀₀₂ , . . . 1120 ₂₅₅ , 1120 ₂₅₆ are small. Therefore, _the calculation results of the nodes of _the second layer 1120 ₀₀₁ , 1120 ₀₀₂ , . Thereby, the calculation cost of the R converter 112r, the G converter 112g, and the B converter 112b can be almost zero. When using a table in the inverse transforming units 115br, 115bg, and 115bb, the output of the inverse transforming unit is set to, for example, 256 gradations, and the numerical value and the output value corresponding to each gradation are set in the table, and the set numerical value is Use the table value closest to , or set the numerical range corresponding to each gradation and the output value for that numerical range in the table, and search in which table value the input data value is included. and the output value may be obtained.

By performing arithmetic processing of the R converter 112r, G converter 112g, and B converter 112b in Embodiment 1 using the conversion table 121, the processing load of the arithmetic processing becomes excessive with a simple configuration. It is possible to provide an information processing apparatus 1A capable of suppressing such a phenomenon and performing processing. Moreover, even if the CNN 114 has few computational resources, the accuracy of machine learning can be improved by using the first converter 112 that can be constructed with few computational resources.

In particular, when the machine learning application of the information processing apparatus 1A of the first embodiment has a heavy processing load, such as super-resolution (improving low-resolution image data to high resolution). , the ratio of the computational cost required for the convolution operation in the entire processing of the CNN 114 is negligibly low. However, when the application of machine learning is light processing load such as image recognition, the ratio of the computational cost required for the convolution operation in the overall processing of the CNN 114 is high. Therefore, it can be said that the reduction of the calculation cost using the conversion table 121 is particularly effective when the computation in the CNN 114 is light.

[Configuration and processing procedure of CNN]
FIG. 7 is a block diagram and a time chart schematically showing the configuration and processing procedure (data processing procedure) of CNN 114 of information processing apparatus 1A of the first embodiment.

As shown in FIG. 7, the CNN 114 includes an input unit 1140 to which data is input, an output unit 1147 to which data is output, and a plurality of layers consisting of a convolution layer and a pooling layer. It has five layers of layer 1142 , third layer 1143 , fourth layer 1144 and fifth layer 1145 and one fully connected layer 1146 . These hierarchies are schematic representations of CNN 114 configuration and processing aspects. Note that the number of convolution layers and pooling layers may be more or less than five layers.

In the CNN 114 of Embodiment 1, first, in the first layer 1141, when convolution processing using a filter (not shown) is performed in the convolution layer ₁₁₄₁₁ , the features of the image data (the image displayed in the image data and graphic features) are extracted, and image data whose size in the two-dimensional direction is reduced from that of the original image data are generated for the number of filters. The pooling layer ₁₁₄₁₂ generates image data in which the size of the image data generated in the convolutional layer is reduced in the two-dimensional direction.

In FIG. 7, the convolution layer 1141 ₁ of the first layer 1141 generates 64 convolution data using 64 types of filters, and the pooling layer 1141 ₂ reduces the two-dimensional size of the 64 types of convolution data. new image data is generated. In the second layer 1142, the convolution layer _1142-1 performs convolution processing using 128 types of filters on the 64 types of image data generated in the first layer 1141 to generate 128 types of convolution data. ₂ , new image data is generated by reducing the two-dimensional size of the 128 types of convolution data.

The same processing is performed for the third layer 1143, the fourth layer 1144, and the fifth layer 1145 thereafter. In the third layer 1143, 256 kinds of convolutional data and new image data are generated by the processing of the convolutional layer 1143 ₁ and the pooling layer 1143 ₂ . In the fourth layer 1144 and the fifth layer 1145, 512 types of convolution data and new image data are generated by processing in convolution layers 1144 ₁ and 1145 ₁ and pooling layers 1144 ₂ and 1145 ₂ .

The fully connected layer 1146 converts the data processed by the first layer 1141 to the fifth layer 1145 into primary data, and recognizes the features of the image displayed in each image data. The fully connected layer 1146 may perform ReLU (Rectified Linear Unit) activation function processing and processing using batch normalization. However, the fully connected layer 1146 may perform processing using any activation function other than ReLU.

[Learning Procedure of Information Processing Device]
In the information processing apparatus 1A of the first embodiment, the image processing execution unit 101 uses the first converter 112, the second converter 113, and the inverse converter 115 as part of the CNN including the CNN 114. do the learning. Specifically, during learning, the image processing execution unit 101 performs processing to minimize an error between output data obtained by inputting learning data to the entire CNN 114 and classification (output) of known learning data. The weights in one transformer 112, second transformer 113, or inverse transformer 115 are updated. The parameters in the CNN 114 and the weights in the first converter 112 and the second converter 113 obtained by this learning process are stored in the storage unit 12 as corresponding parameters. When using the trained CNN 114, the image processing execution unit 101 stores the definition information defining the CNN 114 and the parameters stored in the storage unit 12, and the corresponding first converter 112 and second converter 113 weights are used, and the input data is input to the first converter 112 and the second converter 113, and then the data is input to the CNN 114 and used. When the inverse transformer 115 is used, definition information and parameters that define the learned CNN 114 obtained by learning and corresponding weights are used.

By inputting the first converter 112 and the second converter 113 before the CNN 114 performs feature extraction by convolution, the features of the image data to be extracted can be further emphasized. This is expected to improve the learning efficiency and learning accuracy in the CNN 114 .

[Other configurations]
Note that the communication unit 13, the display unit 14, the operation unit 15, and the reading unit 16 are not essential in the hardware configuration of the information processing apparatus 1A according to the first embodiment. For example, when the communication unit 13 acquires the image processing program 1P, the CNN library 1L, and the converter library 2L stored in the storage unit 12 from an external server device (not shown) or the like, after downloading them once, May not be used. Similarly, the reading unit 16 may be configured not to be used after the image processing program 1P, the CNN library 1L, and the converter library 2L are read from an external storage medium (not shown) and acquired. Also, the communication unit 13 and the reading unit 16 may be the same device using serial communication such as USB (Universal Serial Bus).

Also, the configuration of the information processing apparatus 1A may be distributed over a network (not shown). For example, the above-mentioned CNN 114, the first converter 112, the second converter 113, and the functions as the inverse converter 115 are provided on a Web server (not shown) on a network (not shown), and the display unit and a web client device (not shown) having a communication unit to use these functions. In this case, the communication unit 13 is used to receive a request from a web client device (not shown) and transmit the processing result.

As for the error used during learning, an appropriate function may be used according to the input/output data and the purpose of learning, such as a squared error, an absolute value error, or a cross entropy error. For example, if the output is a classification, use the cross-entropy error. Flexible operation such as using other criteria can be applied regardless of using the error function. The error function itself may be evaluated using an external CNN (not shown).

[Effect]
The information processing apparatus 1A of the first embodiment can easily perform appropriate correction when performing non-linear correction on input data or signals.

This is because the information processing apparatus 1A of the first embodiment is provided with the second transformer 113 and the inverse transformer 115 before and after the CNN 114, and non-linearly spatially transforms the data input to the information processing apparatus 1A. In addition, the first converter 112 is provided in the preceding stage of the second converter 113, and by individually performing non-linear processing on the R data, G data, and B data constituting the image data, the features of the input image data are obtained. can be increased.

With this configuration, the information processing apparatus 1A of the first embodiment increases the features of machine learning in the non-linear conversion of the first converter 112, increases the machine learning recognition rate, or increases the recognition rate of the machine learning. It becomes possible to perform fine image formation.

The processing of the information processing apparatus 1A of the first embodiment may be, for example, a case of performing processing such as gamma correction on color image data in the RGB color space.

For example, for image data having R, G, and B parameters for each pixel, at least one of the R value, G value, and B value, for example, non-linear transformation correction such as gamma correction to the R value (corrections such as individual color space conversions) and non-linear conversion corrections such as gamma corrections to the overall RGB values, any one of the converters that make up the first converter 112; For example, the R converter 112r can be used to nonlinearly transform the R values in the image data, and the second converter 113 can be used to nonlinearly transform the entire RGB values.

By performing such processing, correction such as non-linear transformation is performed on some parameters (for example, the R parameter of RGB) among the plurality of parameters constituting the image data, and all of the plurality of parameters are corrected. It becomes possible to perform correction such as non-linear conversion of . This makes it possible to easily perform multifaceted and accurate correction of data such as image data and signals.

In particular, good conversion results can be obtained by sequentially performing conversion such as nonlinear conversion for data having a plurality of parameters or data of specific parameters among signals and conversion such as nonlinear conversion for data of all parameters. It is considered that the configuration of the first embodiment is highly effective in the case of

The number of convolution layers and pooling layers in CNN 114 is increased, the number of convolution channels (convolution number) is increased, and when the processing load in CNN 114 is increased, the first converter 112 is used. There is a tendency that the improvement of machine learning recognition rate by nonlinear processing for each channel (such as nonlinear processing performed individually for R data, G data, and B data) does not reach the level expected. Therefore, it is considered that the information processing apparatus 1A of the first embodiment is highly effective when the computation in the CNN 114 is light. That is, the information processing apparatus 1A of the first embodiment improves the accuracy of machine learning by using the first converter 112 that can be constructed with few computational resources even if the computational resources in the CNN 114 are small. can be made

In the information processing apparatus 1A of the first embodiment, the first converter 112 is provided with a processing layer group including at least three layers of processing groups of the R converter 112r, the G converter 112g, and the B converter 112b, The second inverse transforming unit 115b has a processing layer group consisting of at least three processing layers of an R inverse transforming unit 115br, a G inverse transforming unit 115bg, and a B inverse transforming unit 115bb, and these processing layer groups are , an input layer with one node, a second layer that is a convolutional layer or dense layer with a plurality of nodes provided after the input layer, and a node number of one provided after the second layer. As a processing layer group including the third layer, which is the convolutional layer or the dense layer, is provided for each channel of data (R, G, B three color channels) to be input to the convolutional neural network, so that a plurality of channels For data having a plurality of parameters, nonlinear processing can be performed for each channel and for each parameter, and the accuracy of machine learning can be further improved.

In the information processing apparatus 1A according to the first embodiment, the second layers of the first converter 112 and the second inverse conversion unit 115b are composed of a plurality of layers, so that multi-color channels such as R, G, and B color channels can be processed. It is possible to further improve the accuracy of machine learning for channel data.

Information processing apparatus 1A of the first embodiment uses second converter 113 to convert data having a plurality of parameters such as R value, G value, and B value into a plurality of parameters (RGB three values). In all cases, for example, the R value and the G value of the three RGB values) are also processed to perform nonlinear conversion, so that nonlinear processing with variations can be easily performed. , the accuracy of machine learning can be further improved.

The information processing apparatus 1A of the first embodiment combines the first converter 112 and the second converter 113 to perform nonlinear conversion, thereby easily performing nonlinear processing with variations. can be done.

The information processing apparatus 1A of the first embodiment performs non-linear conversion using the conversion table 121, thereby reducing the processing load and performing highly accurate machine learning.

The information processing apparatus 1A of the first embodiment is provided with the image processing execution unit 101 that learns the parameters in the convolutional neural network based on the results of the convolution processing. The results can be used to perform highly accurate machine learning.

[Modification]
Note that the information processing apparatus 1A of the first embodiment can also be configured as a modified example shown below. By adopting these configurations, it is possible to perform highly accurate machine learning in an appropriate mode according to the content of data and the content of processing.

(Modification 1)
The number of channels on the output side of the first converter 112 and the second converter 113 provided in the preceding stage of the CNN 114 can be made equal to or greater than the number of channels on the input side. For example, outputs of two or more channels may be obtained in the output layer of the R converter 112r of the first converter. The G converter 112g and the B converter 112b can also have the same configuration. As a result, the RGB 3-channel data input to the first converter 112 is output as 4-channel or more data.

(Modification 2)
The number of channels in the middle of the first converter 112 and the second converter 113 provided in the preceding stage of the CNN 114 can be made equal to or greater than the number of channels on the input side. For example, data can be sent from the first layer 112r1 of the R converter 112r to a second layer (not shown) in a system different from the illustrated second layer 1120 ₀₀₁ , . . . 1120 ₂₅₆ . The G converter 112g and the B converter 112b can also have the same configuration. As a result, the input RGB 3-channel data can be processed as 4-channel or more data in the first converter 112 .

(Modification 3)
The intermediate processing layers of the first converter 112 and the second converter 113 provided in the preceding stage of the CNN 114 can be multi-layered. For example, the intermediate processing layers _{of the R converter 112r of the first converter 112 are arranged after or before the second layers 1120 001 ,} . Another continuous node can be provided before and after each node of the layer. The G converter 112g and the B converter 112b can also have the same configuration.

(Modification 4)
The number of channels on the input side of the inverter 115 provided after the CNN 114 can be made greater than the number of channels on the output side. For example, the data input to the inverse transformer 115 can be 4 channels or more, and the output data can be 3 channels of RGB.

(Modification 5)
The number of channels in the intermediate processing layer of the inverse transformer 115 provided in the subsequent stage of the CNN 114 can be made greater than or equal to the number of channels on the input side (the configuration of the above (Modification 2) is the first inverse transform of the inverse transformer 115 The configuration is applied to the unit 115a and the second inverse transform unit 115b.).

(Modification 6)
The intermediate processing layers of the inverse transformer 115 provided after the CNN 114 can be multi-layered. (The configuration described above (Modification 3) is applied to the first inverse transform unit 115a and the second inverse transform unit 115b of the inverse transform unit 115.).

(Modification 7)
At least one of the R converter 112r, G converter 112g, and B converter 112b of the first converter 112 may be multi-channel input or multi-channel output instead of single-channel input/single-channel output. can. For example, the first layer 112r1 and the third layer 112r3 of the R converter 112r can be configured as two or more nodes. Even with this configuration, the function of the first converter 112 shown in FIG. realizable. However, only when the input side (first layers 112r1, 112g1, 112b1) is one channel, calculations using the conversion table 121 are practically possible.

(Modification 8)
The number of channels on the input side and the number of channels on the output side of the second converter 113 may not be the same as the original number of channels. For example, the first layers 1131r, 1131g, 1131b and the third layers ₁₁₃₃₁ , ₁₁₃₃₂ , ₁₁₃₃₃ of the second transducer 113 may have more or less than three channels. That is, the number of channels may be greater or less than the three RGB channels of the image data input to the input unit 111 .

(Modification 9)
The second layer of the first converter 112 and the second layer of the second inverse converter 115b may be one layer. By configuring in this way, it is possible to reduce the processing load and improve the processing speed.

(Modification 10)
The skip connection applied to the first transformer 112 as shown in FIG. Also, the number of streams of the skip connection is not limited to 1. One processing output of each intermediate processing layer is output by the skip connection, and a stream for synthesizing this output with the other processing output of the intermediate processing layer, and may be composed of a plurality of streams such as a stream for synthesizing the data and the output of the intermediate processing layer.

The configurations of (Modification 1) to (Modification 10) are also applicable to the following [Embodiment 2 of the invention] to [Embodiment 8 of the invention].

[Embodiment 2 of the invention]
FIG. 8 is a functional block diagram showing the configuration of first converter 112 of information processing apparatus 1B according to Embodiment 2 of the present invention.

The information processing apparatus 1B of the second embodiment is applied when it is desired to increase the accuracy even if the amount of calculation is increased.

Specifically, in the information processing apparatus 1B of the second embodiment, the basic configuration of the first converter 112, the second converter 113, the CNN 114, and the inverse converter 115 is the information of the first embodiment. Although it is the same as the processing device 1A ( _{see FIG} . 2), the number of nodes in each of the second layers 1120 ₀₀₁ , 1120 ₀₀₂ , .

The number of _nodes in the second layer 1120 ₀₀₁ , 1120 _{002 ,} . The same applies to the first converter 112 of the information processing device 1B, the first inverse converter 115a, and the second inverse converter 115b of the inverse converter 115 (see FIG. 3). Also, such adjustment of the number of nodes can be similarly applied to all the embodiments of the present invention other than the second embodiment.

In the second embodiment, input data can be processed with high accuracy.

[Embodiment 3 of the invention]
FIG. 9 is a functional block diagram showing part of the image processing section 11 of the information processing apparatus 1C according to Embodiment 3 of the present invention. The image processing unit 11 of this information processing apparatus 1C has the same configuration as that of the information processing apparatus 1A of the first embodiment except that the second converter 113 does not exist. In this case, the inverse converter 115 can be configured without the first inverse converter 115 a corresponding to the second converter 113 .

With such a configuration, it is possible to perform appropriate processing when there is no need to perform nonlinear processing by spatial transformation using a plurality of parameters at once.

[Embodiment 4 of the invention]
FIG. 10 is a functional block diagram showing part of the image processing section 11 of the information processing apparatus 1D according to Embodiment 4 of the present invention. The image processing unit 11 of this information processing apparatus 1D has the same configuration as that of the information processing apparatus 1A of the first embodiment except that the inverse transformer 115 is not present.

Such a configuration is used when the output data does not require nonlinear transformation processing.

As a modification of the information processing apparatus 1D of the fourth embodiment, one or more of the R inverse transforming section 115br, the G inverse transforming section 115bg, and the B inverse transforming section 115bb of the information processing apparatus 1A of the first embodiment. A configuration in which the two do not exist is also possible.

[Embodiment 5 of the invention]
FIG. 11 is a functional block diagram showing part of the image processing section 11 of the information processing apparatus 1E of the fifth embodiment. The image processing unit 11 of this information processing apparatus 1E is the same as the information processing apparatus 1A of Embodiment 1 except that the second converter 113 and the inverse converter 115 are not present.

Such a configuration is used when the output data does not require nonlinear transformation processing.

[Embodiment 6 of the invention]
FIG. 12 is a functional block diagram showing part of the image processing section 11 of the information processing apparatus 1F according to the sixth embodiment. The image processing unit 11 of the information processing apparatus 1F differs from the information processing apparatus 1A of the first embodiment in that the first converter 112 and the second converter 113 are connected in reverse. Although not shown, the first inverse transforming unit 115a and the second inverse transforming unit 115b that constitute the inverse transforming unit 115 may be connected in reverse to the information processing apparatus 1A of the first embodiment.

By configuring in this way, when it is desired to perform spatial processing by the second converter 113 first and emphasize the spatial processing, or when processing individual parameters by the first converter 112 is performed later and Appropriate processing can be performed when, for example, it is desired to emphasize processing. Note that the information processing apparatus 1F may be configured without the inverter 115. FIG.

[Embodiment 7 of the invention]
FIG. 13 is a functional block diagram showing part of the image processing section 11 of the information processing apparatus 1G of the seventh embodiment. The image processing unit 11 of the information processing apparatus 1G does not include the inverse transformer 115 of the information processing apparatus 1F of the sixth embodiment. With such a configuration, appropriate processing can be performed on data to be appropriately processed by the information processing apparatus 1F according to the sixth embodiment when inverse transformation is not required.

[Embodiment 8 of the invention]
Although not shown, in the information processing apparatus of this embodiment, in the configuration of the information processing apparatus 1A of Embodiment 1, both the first converter 112 and the second converter 113 are placed before the CNN 114. A configuration in which they are not provided and/or a configuration in which the first converter 112 and the second converter 113 are provided after the CNN 114 is also possible.

It goes without saying that the above embodiments are examples of the present invention, and the present invention is not limited only to the above embodiments.

[Example]
Examples of the present invention will be described below.

FIG. 14 shows an embodiment of the invention. FIG. 14A is a functional block diagram showing a part of the configuration of the image processing section 11 as Conventional Example 1. FIG. The image processing unit 11 directly inputs the input data to the CNN 114 .

FIG. 14B is a functional block diagram showing a part of the configuration of the image processing section 11 as Conventional Example 2. As shown in FIG. In this image processing unit 11 , the input data is input to the second converter 113 and then to the CNN 114 .

FIG. 14C is a functional block diagram showing part of the configuration of the image processing section 11 as the present invention. In this image processing unit 11 , the input data is input to the first converter 112 and then to the CNN 114 .

In this embodiment, an experiment was conducted to make the image processing unit identify image data showing ten kinds of pictures (airplane, car, bird, cat, deer, dog, frog, horse, ship, and truck). Specifically, after having the image processing unit learn the 10 types of pictures described above, the image processing unit is caused to read an image to be recognized and determine which of the 10 types of pictures the read image corresponds to. After recognition, an experiment was conducted in which a symbol corresponding to each picture was output and an answer was given.

In this experiment, a modified version of VGG16 was used as the machine learning model, CIFAR-10 was used as the data set, the number of correct answers was obtained with respect to the number of pictures read, and the validity accuracy (percentage of correct answers) (%) was calculated. verified.

As shown in FIG. 14, each image processing unit 11 is not provided with an inverse converter. This is because it is configured to output symbols in response to the input of image data, and it was thought that the recognition accuracy would be lowered if an inverse transformer was provided.

この表に示すとおり、従来例１、従来例２に比べ、本件発明は改善された正答率が得られている。よって、本件発明は、従来例に比べて高い認識率が得られることがわかる。なお、正答率の改善は１％未満と僅かではあるが、機械学習においては僅かであっても正答率を向上させることは重要な課題である。 The results of the experiments are shown in the table below.

As shown in this table, compared with Conventional Examples 1 and 2, the present invention provides an improved percentage of correct answers. Therefore, it can be seen that the present invention can obtain a higher recognition rate than the conventional example. Although the improvement in the correct answer rate is less than 1%, it is an important issue in machine learning to improve the correct answer rate even if it is only a little.

1A, 1B, 1C, 1D, 1E, 1F, 1G, 1H, 1J, 1K... Information processing device 12... Storage section (storage means)
121: conversion table 101: image processing execution unit (learning execution unit)
112 First converter (converting means, first nonlinear processing means)
113 second converter (conversion means, second nonlinear processing means)
114 CNN (data processing means)
115 Inverse converter (inverse conversion means)
112r1, 112g1, 112b1, 1131r, 1131g, 1131b, 115a11, _115a12 , _115a13 , _115br1 , 115bg1, 115bb1... First layer (input layer)
1120 _{001 ,} 1120 ₀₀₂ , _{. .} . 1120 _{255 ,} 1120 ₂₅₆ , 1132 ₀₀₁ , . 5br2 ₀₀₂ , _{... 115br2 255 , 115br2 256} , _{115bg2 001} , _{115bg2 002 , .}
112r3, 112g3, 112b3, 1133r, 1133g, 1133b, _115a31 , 115a32, _115a33 , _115br1 , 115bg3, 115bb3... third layer (output layer)

Claims (9)

An information processing device comprising a convolutional neural network including a convolutional layer and comprising data processing means for performing convolution processing on data having a plurality of channels,
conversion means for performing non-linear conversion on data input to the information processing device and inputting the data to the data processing means;
and/or
An inverse transformation means for performing non-linear transformation on the data output from the data processing means and outputting it from the information processing device,
The information processing apparatus, wherein the transforming means and/or the inverse transforming means comprises first nonlinear processing means for individually performing the nonlinear transformation on the data for each channel. The transforming means and/or the inverse transforming means,
A treated layer group consisting of at least three treated layers,
The processing layer group includes an input layer having one node, an intermediate processing layer having a plurality of nodes or a dense layer provided after the input layer, and an intermediate processing layer provided after the intermediate processing layer. and an output layer that is a convolutional layer or a dense layer with one or more nodes,
2. The processing apparatus according to claim 1, wherein said processing layer group is provided for each channel of said data input to said convolutional neural network. 3. A processing apparatus according to claim 2, wherein said intermediate processing layer consists of one layer. 3. A processing apparatus according to claim 2, wherein said intermediate processing layer comprises a plurality of layers. 5. The transforming means and/or the inverse transforming means according to claim 1, further comprising a second nonlinear processing means for performing the nonlinear transformation by combining a plurality of the channels. 1. The information processing device according to one. A storage means for storing a conversion table in which conversion modes used in the first nonlinear processing means are recorded,
6. The information processing apparatus according to claim 1, wherein said first non-linear processing means performs said non-linear conversion using said conversion table obtained from said storage means. 7. An information processing apparatus according to claim 1, wherein said transforming means and/or said inverse transforming means uses a skip connection. An information processing method in an information processing device, comprising a data processing procedure in which convolution processing is performed on data having a plurality of channels in a convolutional neural network including a convolution layer,
a transformation procedure for performing non-linear transformation on data input to the information processing device and inputting the data processing procedure;
and/or
An inverse transformation procedure for performing non-linear transformation on the data output by the processing of the data processing procedure and outputting it from the information processing device,
The information processing apparatus, wherein the transforming procedure and/or the inverse transforming procedure include a first nonlinear processing procedure in which the nonlinear transformation is performed on the data separately for each channel. Information processing methods. A program that causes a computer to function as the information processing apparatus according to any one of claims 1 to 7.

Images