JP2020052689A - Image processing system - Google Patents

Abstract

To enable efficient learning.SOLUTION: An image processing system for performing image recognition processing by using a neural network (NN), includes first processing means for performing a first process by using an NN at a preceding stage, second processing means for performing a second process by using an NN at a succeeding stage, learning means for learning parameters including a coefficient of connecting each node of an output layer of the first processing means to each node of an input layer of the second processing means on the basis of an output value from the output layer of the first processing means, and determination means for determining a dropout node at which transmission of the output value from each node of the output layer of the first processing means to the learning means is suppressed in accordance with a communication band between the first processing means and the learning means.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a multilayer neural network.

  In recent years, a neural network having a multilayer structure called a deep net or a deep neural network, and deep learning (also referred to as deep learning), which is a method for learning the network, have attracted a great deal of attention in recent years. Although a deep net does not refer to a specific calculation method, generally, a hierarchical processing (a processing result of a certain layer is applied to input data (for example, an image), ).

  In the field of image identification, a deep net composed of a convolution layer for performing a convolution filter operation and a fully connected layer for performing a full connection operation is becoming mainstream. Patent Literature 1 discloses a technique in which in a character recognition process, a training unit and a recognition unit are configured separately and a feature vector is classified by a support vector machine (SVM).

  In addition, Non-Patent Literature 1 and Non-Patent Literature 2 refer to a technique called “dropout” that contributes to data reduction during learning in a deep net. For example, in a forward-propagation type neural network as shown in FIG. 1, when learning the coupling coefficient of the solid line portion by the error back propagation method, a dropout node is determined from a node represented by a thick circle. Then, the dropout node always sets its output to 0. Here, the dropout node is determined for each node at a predetermined probability called a dropout rate. For example, if the dropout rate is 0.5, each node becomes a dropout node with a probability of 0.5. With this configuration, it has been confirmed that overfitting to learning data is suppressed, and the performance as a classifier is improved.

JP 2009-48641 A

A. Krizhevsky, I. Sutskever and G.E.Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", NIPS, 2012 G.E.Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever and R.R.Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors", CoRR, 2012.

  However, in Patent Literature 1, learning cannot be performed properly if the bandwidth of a communication path connecting the training unit and the feature vector extraction unit is not sufficient. Further, in the dropouts disclosed in Non-Patent Documents 1 and 2, since the dropout rate is given, appropriate learning should be performed when the bandwidth of the communication path changes with time. Can not do. Further, in Non-Patent Documents 1 and 2, each node independently determines a dropout node based on a dropout rate. Therefore, the number of dropout nodes cannot be controlled to match the target number. As a result, when there are many dropout nodes, learning cannot be performed by effectively utilizing the bandwidth of the communication path, and when there are few dropout nodes, sufficient learning cannot be performed.

  The present invention has been made in view of such a problem, and has as its object to provide a technology that enables efficient learning.

In order to solve the above problems, an image processing system according to the present invention has the following configuration. That is, an image processing system that performs image identification processing using an L-layer neural network (NN)
First processing means for performing a first process using the NN of the preceding M layer of the L layer;
A second processing unit that performs the second processing using the NN of the subsequent (LM) layer of the L layer;
A parameter including a coupling coefficient from each node of the output layer of the first processing unit to each node of the input layer of the second processing unit based on an output value from an output layer of the first processing unit. Learning means to learn,
A dropout node for suppressing transmission of an output value from each of the nodes of the output layer of the first processing means to the learning means in accordance with a communication band between the first processing means and the learning means Determining means for determining
Has,
The first processing means transmits, to the learning means, output values from each node of the output layer of the first processing means excluding the dropout node determined by the determining means.

  According to the present invention, a technology that enables efficient learning can be provided.

FIG. 3 is a diagram illustrating a dropout technique. FIG. 1 is a diagram illustrating a configuration of a system according to a first embodiment. It is a flowchart of a determination process of a dropout rate. FIG. 3 is a diagram illustrating an example of an arrangement of a data extension unit in the system. It is a figure which shows a series of processes of a deep net exemplarily. It is a flowchart of the determination processing of a dropout node. It is a figure explaining composition of an image identification system in a 5th embodiment. It is a figure showing an example of arrangement of a data extension part in an image identification system. It is a figure explaining composition of an image identification system in a 6th embodiment. It is a flowchart of a relearning process. It is another flowchart of a relearning process. It is a figure explaining composition of an image identification system in an 8th embodiment. It is a figure which shows a series of processes of a deep net exemplarily. It is a flowchart of the re-learning process in the learning data distribution unit.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings. The following embodiments are merely examples, and are not intended to limit the scope of the present invention.

(1st Embodiment)
As a first embodiment of an image processing system according to the present invention, a system using a neural network will be described below as an example.

<System using multiple housings>
When the deep net is viewed as an image identification algorithm installed in a surveillance camera, etc., it is considered that layers close to the input are performing feature extraction, and layers close to the output, especially fully connected layers, are based on extracted features. It is considered that identification processing is being performed. It is generally known that the performance of the feature extraction processing does not change much when the camera installation environment or the discrimination target field such as a person or a car is different from that at the time of learning, but the performance of the discrimination processing is greatly improved by relearning.

  Here, a series of processing of image identification by the deep net is divided into two by a layer break, and processing closer to an input is defined as a first processing, and processing closer to an output is defined as a second processing. In the first process, learning of a network is performed in advance using typical data in an installation environment in which a monitoring camera is used or an identification target field, and after installation, only the network in the second process is learned.

  From the above-mentioned knowledge, it can be expected that even in the case of such a configuration, the image identification performance does not decrease much compared to the case where both processes are learned after installation. Thus, by learning the network only for the processing part that greatly affects the discrimination performance, the learning cost can be reduced without significantly lowering the discrimination performance. By providing the learning function in a separate housing from the surveillance camera body and including the learning function in the installation work or as a paid or free service, the price of the surveillance camera body can be reduced.

  The learning of the network of the second processing is performed from the part of performing the first processing (hereinafter, referred to as a first processing unit) to the part of learning the network of the second processing (hereinafter, referred to as a second processing learning unit). ) Is transmitted by sending a large amount of data.

  In the surveillance camera system having the above-described configuration, the first processing unit and the second processing learning unit are housed in different housings, and thus the large amount of data described above is sent via a communication path connecting these. However, the communication band of the communication path between different housings is generally narrower than the communication band of the communication path inside the housing. Therefore, in learning the second network, it is necessary to reduce the amount of data sent from the first processing unit to the second processing learning unit.

<System configuration>
FIG. 2 is a diagram illustrating the configuration of the system according to the first embodiment. The system 2 includes a first housing 21 and a second housing 22. The first housing 21 and the second housing 22 are configured to be able to communicate with each other via a communication path 23 such as a communication network using respective communication units (the communication unit 211 and the communication unit 221). ing. In the following description, data transfer between each unit in the first housing 21 and each unit in the second housing 22 is performed by the communication unit 211, the communication path 23, and the communication unit 23, unless otherwise specified. This is performed via the unit 221.

  The first housing 21 includes an image acquisition unit 212, a pre-processing unit 213, a first processing unit 214, and a data transfer control unit 215. The second housing 22 includes a band estimation unit 222 and a second processing learning unit 223. Further, the second processing learning unit 223 includes a dropout rate determination unit 2231. Here, it is assumed that each functional unit is realized by the CPU executing a predetermined program. However, one or more functional units may be realized using hardware such as an ASIC.

  The image acquisition unit 212 acquires a digital image using a CCD, a CMOS sensor, or the like, and transfers the digital image to the preprocessing unit 213. The image acquisition unit may be any unit that can provide a digital image, and may be, for example, a unit that reads a digital image recorded on a storage medium. For example, a digital image stored in a video server or an image server may be read using a communication unit (not shown).

  The preprocessing unit 213 performs processes such as color tone correction, brightness correction, monochrome conversion, and edge enhancement depending on the application, and may be omitted depending on the application. The processes such as color tone correction, brightness correction, monochrome conversion, and edge enhancement performed by the preprocessing unit 213 are known in the field of image processing, and thus the details are omitted.

  The first processing unit 214 performs the first processing as described in the background art. For example, the first process is performed using the NN of the M layer preceding the L layer neural network (NN). Then, the output values of all output nodes in the first processing are sent to the data transfer control unit 215.

  The data transfer control unit 215 converts only the output values of the nodes that have not been dropped out, designated by the second processing learning unit 223, from among the output values of all the output nodes received from the first processing unit 214, in the second processing Hand over to learning unit 223. In the following description, the output value of the above-described node passed from the data transfer control unit 215 to the second processing learning unit 223 is referred to as “learning data”.

  The second processing learning unit 223 learns a coupling coefficient between at least the last layer of the first processing and the first layer of the second processing from the learning data received from the data transfer control unit 215 by the error back propagation method. I do. Note that the second process is a process executed using the NN of the (LM) layer subsequent to the L-layer neural network (NN). Note that the second processing learning unit 223 may further learn the coupling coefficient and the like inside the second processing.

  If the corresponding correct answer is known in advance, the teacher data used for learning may be used. For example, when the image acquisition unit 212 is a recorded video server or image server, it is possible to create teacher data in advance. If the correct answer is not known, for example, the result obtained by performing forward propagation evaluation on the network being learned by the second processing learning unit 223 may be used as the estimated value.

  In the second processing network where learning has progressed to some extent, in many cases, it can be expected that the estimated value and the correct answer match. The method of re-learning when the estimated value differs from the correct answer will be described in another embodiment described later.

  The band estimating unit 222 inquires of the communication unit 221 about the communication status, and estimates the communication band of the communication path 23. As a band estimation method, any known method can be used. For example, "Kenji Kugimoto, Shigeki Miyake, Tomohiro Mizutani," Available Bandwidth Measurement Function of TCP with Socket API ", Computer Software Vol.31 (2014) No.3 p.3_246-3_258, Japan Society for Software Science and Technology" The disclosed approach is available. The band estimation result of the communication channel 23 is passed to the dropout rate determination unit 2231.

  The dropout rate determination unit 2231 determines the dropout rate of the final layer of the first processing from the estimated communication band of the communication path 23. When mini-batches are used by the stochastic gradient descent method (SGD: Stochastic Gradient Descent), the dropout rate may be changed in units of mini-batches. The method for determining the dropout rate will be described later with reference to FIG.

  The second processing learning unit 223 determines a dropout node that suppresses transmission of an output value based on the dropout rate determined by the dropout rate determination unit 2231. Then, it instructs the data transfer control unit 215 which node to pass to the second process learning unit 223 as learning data.

<System operation>
FIG. 3 is a flowchart of the dropout rate determination process. This processing is executed by the dropout rate determination unit 2231.

  In step S31, the dropout rate determination unit 2231 acquires the estimated communication band of the communication path 23 from the band estimation unit 222. In step S32, the dropout rate determination unit 2231 calculates a minimum dropout rate p_dropout_min that does not compress the communication path 23. For example, the minimum p_dropout_c that satisfies the following equation (1) may be set as p_dropout_min.

(Equation 1)
datasize_node * nodes_firstlayer * samplerate * (1-p_dropout_c) ≤ bandwidth ・ ・ ・ (1)

here,
datasize_node: data size [Bytes] required to send the output per node of the first processing output node;
nodes_firstlayer: number of all output nodes of the first process;
samplerate: learning sample rate (number of learning samples to be processed per second) [samples / sec];
p_dropout_c: dropout rate;
bandwidth: estimated communication bandwidth [Bytes / sec];
It is.

  It is preferable to actually consider a header necessary for communication, an additional data amount due to data alignment, and the like. These are values that can be estimated in advance, depending on the communication protocol used by the communication unit 211 and the communication unit 221 and the data structure used to transmit the data of the output node.

  In step S33, the dropout rate determination unit 2231 determines a dropout rate p_dropout for the next mini-batch. For example, it may be determined as shown in Expression (2). That is, the larger of the dropout rate p_dropout_best considered to be the best when the communication band of the communication path 23 is not taken into consideration and the minimum dropout rate p_dropout_min that does not compress the communication path 23 may be set to p_dropout. The value of p_dropout_best can be determined arbitrarily, but according to Non-Patent Document 2, for example, 0.5 is considered to be good.

(Equation 2)
p_dropout: = max (p_dropout_best, p_dropout_min) (2)

  In step S34, the dropout rate determination unit 2231 outputs the dropout rate p_dropout calculated in S33. In order to perform learning more effectively, a technique called data extension may be used. This is a technique in which learning is performed on an image obtained by changing the trimming, rotation, contrast, and the like of an input image. Accordingly, learning is performed even on samples several times the number of input samples (= input image number), and a highly accurate network can be learned.

  FIG. 4 is a diagram illustrating an example of an arrangement of a data extension unit in the system. Here, an example is shown in which the data expansion unit 411 is arranged between the preprocessing unit 213 and the first processing unit 214. When the learning data is increased by n times due to data expansion, the condition of Expression (1) is Expression (3) in which the left side is multiplied by n.

(Equation 3)
datasize_node * n * nodes_firstlayer * samplerate * (1-p_dropout_c) ≤ bandwidth ・ ・ ・ (3)

  As described above, according to the first embodiment, the node to be dropped out is set as the output node of the last layer of the first processing. With this configuration, the amount of learning data passed from the first processing unit to the second processing learning unit is significantly reduced. Therefore, it is possible to suitably perform learning even in a configuration in which the first processing unit and the second processing learning unit are arranged in different housings.

  Further, since the dropout rate is determined in consideration of the communication band of the communication path, learning can be efficiently performed even if the band of the communication path is relatively narrow. Further, even when the band of the communication path changes with time, learning suitable for the communication band can be performed.

(2nd Embodiment)
In the second embodiment, a method of determining a position (layer) at which the first processing 511 and the second processing 512 are separated will be described.

  FIG. 5 is a diagram exemplarily showing a series of processing of the deep net. FIG. 5 is an example, and any neural network having a multilayer structure may be used. In FIG. 5, the left end is an input layer and the right end is an output layer, which is a network of nine layers (L = 9). Therefore, eight layers 51 to 58 exist.

  First, when one image is input at the left end, the amount of data passing through each of the layers 51 to 58 is estimated. The method of estimating the data amount can be calculated by multiplying the output data width per node by the number of interlayer connections. In a fixed neural network, the amount of data passing through each interlayer connection is predictable.

  Next, the estimated data amount is compared, and a layer that satisfies “Condition 1: closer to the input and smaller than the amount of data flowing between any layers” is searched. For example, if the amount of data flowing between the layers 56 is smaller than any of the amounts of data flowing between the layers 51 to 55, a series of processes is divided into a first process 511 and a second process 512 at the position of the layers 56. . FIG. 5 shows an example in which the first processing is performed using NNs of six layers (M = 6) at the preceding stage of the neural network (NN) of all nine layers. Then, the second process is performed using the three-layer NN of the former stage.

  By dividing the first processing 511 and the second processing 512 at the positions (layers) determined as described above, the amount of data flowing through the communication path 23 can be reduced. As a result, it can be applied to the communication path 23 having a narrower band.

(Third embodiment)
In the third embodiment, a method for determining a specific dropout node will be described. That is, a method of determining a dropout node based on the dropout rate described in the first embodiment will be described.

  FIG. 6 is a flowchart of the dropout node determination process. This processing is executed by the second processing learning unit 223.

  In step S61, the second processing learning unit 223 acquires the dropout rate p_drouput determined by the dropout rate determination unit 2231.

  In step S62, the second processing learning unit 223 calculates the number of dropout nodes, nodes_dropout. This can be calculated according to equation (4) using the total number of nodes, nodes_firstlayer, of the output layer of the first processing. Note that ceil () is a function representing rounding up.

(Equation 4)
nodes_dropout: = ceil (nodes_firstlayer * p_dropout) ・ ・ ・ (4)

  In step S63, the second processing learning unit 223 determines a dropout node by randomly selecting nodes for the number of nodes dropout_nodes from all output nodes of the first processing.

  In step S64, the second processing learning unit 223 notifies the information of the determined dropout node to the first processing unit 214.

  As described above, by determining the dropout nodes, the number of dropout nodes can be accurately controlled. Therefore, in the learning of the network, the learning can be reliably performed without exceeding the communication band of the communication path 23.

(Fourth embodiment)
In the fourth embodiment, another method for determining a position (layer) at which the first processing 511 and the second processing 512 are separated will be described.

  There is not necessarily one location between the layers satisfying the “condition 1” described in the second embodiment. As described in the background art, in the field of image identification, a deep net composed of a convolution layer for performing a convolution filter operation and a fully connected layer for performing a full connection operation is becoming mainstream. In this case, it is considered that the convolutional layer performs the feature extraction processing and the fully connected layer performs the identification processing.

  Furthermore, it is generally known that the performance of the feature extraction processing does not change much when the camera installation environment or the discrimination target fields such as people and cars differ from those at the time of learning, but the performance of the discrimination processing greatly increases by re-learning. I have.

  Therefore, when the candidate satisfying the “condition 1” described in the second embodiment includes an interlayer between the convolutional layer and the fully connected layer, a series of processing of the deep net is performed on the interlayer (position) in the first layer. Is divided into a process 511 and a second process 512.

  By doing as described above, it is possible to effectively perform learning while suppressing a decrease in identification performance.

(Fifth embodiment)
In the fifth embodiment, an image identification system that performs image identification by the second processing will be described.

<System configuration>
FIG. 7 is a diagram illustrating a configuration of an image identification system according to the fifth embodiment. 2, a second processing unit 711, a recognition result processing unit 712, and a parameter setting unit 713 are added in the first housing 21. Here, only differences from the above-described embodiment will be described.

  The data transfer control unit 215 transfers, to the second process learning unit 223, only the learning data designated by the second process learning unit 223 among the output values of all the output nodes received from the first processing unit 214. Further, the output values of all output nodes received from the first processing unit 214 are passed to the second processing unit 711.

  The second processing unit 711 performs the second processing to perform image identification. The network used in the second processing performed by the second processing unit 711 uses the parameters learned by the second processing learning unit 223.

  Here, the learning of the network is assumed to be the learning of the coupling coefficient of the interlayer coupling by the error back propagation method, and the structure of the network itself does not change. Therefore, the expression "parameter" is used as a learning target. However, the learning target may be the activation function of the node or the network structure itself. The parameters learned by the second processing learning unit 223 are passed to the parameter setting unit 713.

  The parameter setting unit 713 sets parameters for the second processing unit 711 in mini-batch units. Note that the parameters include a coupling coefficient between the first processing and the second processing. The result of the image identification by the second processing unit 711 is passed to the identification result processing unit 712.

  The identification result processing unit 712 performs some processing according to the application area of the image identification system using the result of image identification by the second processing unit 711. For example, a surveillance camera counts the number of people or issues an alert when a suspicious person is shown. Further, an image of the suspicious individual may be sent to a monitor in a security room using a communication unit (not shown).

  FIG. 8 is a diagram illustrating an example of an arrangement of the data extension unit in the image identification system. That is, similarly to the first embodiment (FIG. 4), a configuration in which data extension is performed by the data extension unit 411 can be adopted. In this case, the data transfer control unit 215 performs the above-described operation on the data before data expansion. On the other hand, the data that has been subjected to data expansion is sent to the second processing learning unit 223 as learning data, but may not be sent to the second processing unit 711.

  With the above configuration, the image identification processing is completed in the first housing 21. Therefore, the second housing 22 may be installed only when learning the second process. Therefore, there is an effect that it is easy to reduce the size and the price of the portion (the first housing 21) for performing the image identification processing.

  In addition, learning for a plurality of first housings 21 can be performed using one second housing 22. Therefore, the total system using the plurality of first housings 21 can be reduced in cost.

  Further, the "learning" itself can be used as a service at the time of installation of the image identification system or as a separately paid service, and can correspond to various business models. For example, by arranging all the functional units constituting the second housing 22 on the cloud, the learning part can be a service independent of the first housing 21. Further, it becomes possible to collect data from many first casings 21, and it becomes easy to improve the learning algorithm.

  Further, the data flowing through the communication path 23 is equivalent to the feature amount after the first processing. Therefore, the user's psychological resistance is lower than in the case where the input image is sent as it is, and privacy is considered.

(Sixth embodiment)
In the sixth embodiment, a method will be described in which correct teacher data is added and the second process is re-learned. In the above embodiment, the teacher data was not explicitly given. However, if, for example, an erroneous identification is found while monitoring the image identification result, it is necessary to provide correct teacher data and re-learn the second process.

<System configuration>
FIG. 9 is a diagram illustrating a configuration of an image identification system according to the sixth embodiment. 7, the teacher data correction instructing unit 911 increases in the first housing 21, and the parameter storage unit 921, the learning data storage unit 922, and the teacher data correction unit 923 increase in the second housing 22. . Here, only differences from the above-described embodiment will be described.

  The learning data sent from the data transfer control unit 215 is temporarily stored in the learning data storage unit 922. As will be described later, the learning data is stored together with the corresponding teacher data and information specifying the timing of the learning data. In the following description, the “information for specifying the period of the learning data” is simply referred to as “time”.

  The teacher data only needs to hold at least the corrected teacher data passed from the teacher data correction unit 923. The teacher data correction unit 923 will be described later. In addition, as for the learning data to which the teacher data is not added, the second processing learning unit estimates and corrects the correct answer as in the above-described embodiment, but for the learning data to which the teacher data is added, the estimated value is used. Learning is performed using teacher data without using.

  The information for specifying the timing of the learning data may be, for example, the time at which the original image of the learning data was captured, or the total number of captured images (frame numbers), and may be any value that monotonically increases with time. .

  The learning data once stored in the learning data storage unit 922 is read from the second processing learning unit 223, performs learning as usual, and calculates learned parameters. The calculated learned parameters are not directly sent to the parameter setting unit 713, but are temporarily stored in the parameter storage unit 921 and then sent to the parameter setting unit 713. The learned parameter is stored together with information for specifying the time of the learning data, that is, the time.

  For example, if a parameter is learned using an image captured between time T0 and time T1 as a mini-batch, T1 may be used as the time of the learned parameter.

  When the image identification result is incorrect, the teacher data correction instructing unit 911 inputs the teacher data and information for specifying the time. For example, when the identification result processing unit 712 is monitoring monitor application software, a supervisor watching the monitor screen inputs teacher data when erroneous identification is found. The teacher data and the time are passed from the teacher data correction instruction unit 911 to the teacher data correction unit 923.

  The teacher data correction unit 923 associates the teacher data with the learning data of the period stored in the learning data storage unit 922. In addition, the time is passed to the second processing learning unit 223 to instruct re-learning.

<System operation>
FIG. 10 is a flowchart of the relearning process. This processing is executed by the second processing learning unit 223.

  In step S101, when the re-learning is instructed by the teacher data correction unit 923, the second process learning unit 223 acquires the timing Tt of the teacher data. In step S102, the second process learning unit 223 deletes the learned parameters stored in the parameter storage unit 921 after the time Tt. This is because the learned parameters after that, including the teacher data that triggers the re-learning, are not needed to perform the re-learning.

  In step S103, the second processing learning unit 223 acquires the last learned parameter from the parameter storage unit 921, and sets the second processing network to the initial state. This is because the latest learned network at that time is set to the initial state in order to re-learn the mini-batch including the teacher data that triggers the re-learning.

  In step S104, the second processing learning unit 223 selects a mini-batch that includes the time Tt as a mini-batch to be learned.

  In step S105, the second process learning unit 223 performs network learning while acquiring the learning data of the selected mini-batch from the learning data storage unit. Since this is the same as a known learning process, the details are omitted. At the end of the learning, the learned parameters are stored in the parameter storage unit 921.

  In step S106, the second processing learning unit 223 selects a mini-batch next to the selected mini-batch. If there is a mini-batch, the process returns to S105 and learning of the mini-batch is performed. If there is no mini-batch, that is, if learning of the last mini-batch has been completed, the process proceeds to S107.

  In step S107, the second processing learning unit 223 passes the learned parameter learned last to the parameter setting unit 713, and updates the second processing network of the second processing unit 711. Here, the process of S107 is performed at the end of the re-learning, but may be performed every time after S105.

  As described above, according to the sixth embodiment, it is possible to perform relearning when the image identification result is erroneous identification. By performing the re-learning, it is possible to obtain a more suitable image identification result.

(Seventh embodiment)
In the seventh embodiment, a method for controlling the accumulation amount of learning data and learned parameters will be described. In the above-described embodiment, it has been described that the learning data and the learned parameters are stored in the learning data storage unit 922 and the parameter storage unit 921, respectively, for re-learning.

  However, the more past the learning data and learned parameters are, the less likely they are to be referred to for re-learning, and the amount of data to be accumulated is enormous. That is, it is realistic to discard learning data and learned parameters that are somewhat old. Generally, the data amount of the learning data is orders of magnitude larger than that of the learned parameters.

  Further, when the teacher data correction instruction section 911 instructs to correct the teacher data, in order to eliminate the erroneous learning result, the re-learning should use the learned parameter before the time of the teacher data. For this reason, the learned parameters stored in the parameter storage unit 921 should be retained by the teacher data correction instructing unit 911 even if they are older than the next period that may be instructed. On the other hand, it is realistic to hold only the learning data stored in the learning data storage unit 922 after the time when the possibility that re-learning is performed is high to some extent.

  Therefore, it is desirable that the learned parameter holding period of the parameter storage unit 921 be longer than the learning data holding period of the learning data storage unit 922.

<System operation>
When a new learned parameter is passed from the second processing learning unit 223, the parameter storage unit 921 deletes an old learned parameter that has passed a predetermined holding period. The deletion of the old learned parameter may be periodically executed using a timer (not shown) or the like.

  When new learning data is passed from the data transfer control unit 215, the learning data storage unit 922 deletes old learning data that has passed a predetermined retention period. The deletion of the old learning data may be performed periodically using a timer (not shown) or the like.

  FIG. 11 is another flowchart of the relearning process. This is almost the same as FIG. 10, except that S104 is replaced with S114.

  In step S114, the second processing learning unit 223 selects a mini-batch to be learned that includes the time Tt. However, if the mini-batch does not exist, the process proceeds to S106, where the first mini-batch is selected and re-learning is performed.

  The learned parameter holding period should be set long enough so that there is a learning parameter to be selected in S103, but the oldest learned parameter may not be discarded.

  As described above, according to the seventh embodiment, it is possible to perform re-learning while preventing data accumulated in the system from being enlarged.

(Eighth embodiment)
In the eighth embodiment, a method of dynamically changing the position (layer) at which the first processing 511 and the second processing 512 are separated will be described.

  In the above-described embodiment, in FIG. 5, a series of processes is divided into a first process 511 and a second process 512 between layers 56. Then, a dropout was performed on the node in the final layer of the first processing, and learning was performed. However, if the amount of data flowing between the layers 57 is smaller than the amount of data flowing between the layers 56, a series of processing can be divided between the layers 57.

  Therefore, in the eighth embodiment, the division between the first process 511 and the second process 512 is dynamically switched based on the estimated value of the band of the communication channel 23 by the band estimating unit 222. This is applicable to the system, but also to the image identification system including the system.

<System configuration>
FIG. 12 is a diagram illustrating a configuration of an image identification system according to the eighth embodiment. The circled numbers (1 to 4) in the figure indicate that the corresponding numbers are connected. Compared with FIG. 9, the third processing unit 1211 and the third processing learning unit 1222 are increased, and the third processing determining unit 1221, the learning data distribution unit 1223, and the drop in the third processing learning unit 1222 The out-ratio determination unit 12221 has increased. Here, only differences from the above-described embodiment will be described.

  FIG. 13 is a diagram exemplarily showing a series of processing of the deep net. It is almost the same as FIG. 5, except that the processing is divided into three parts, a first processing 511, a second processing 512, and a third processing 513. When a series of processes is divided between the layers 56, the third process 513 is integrated with the second process 512. On the other hand, in the case of division at the interlayer 57, the third processing 513 is integrated with the first processing 511.

  The first processing unit 214 passes the output values of all output nodes of the first processing to the data transfer control unit 215 as in the above-described embodiment.

  When a series of processes is divided between the layers 56, the data transfer control unit 215 sends the value of the node specified by the third process learning unit 1222 to the learning data storage unit 922. If a series of processing is divided between the layers 57, all values received from the first processing unit are passed to the third processing unit 1211.

  The third processing unit 1211 performs the third processing 513, and passes the result, that is, the output values of all the output nodes of the third processing 513 to the data transfer control unit 215. Then, the value of the node specified by the second processing learning unit 223 is sent to the learning data storage unit 922.

  In the data transfer control unit 215, whether the series of processing is divided between which layers is determined by whether the node instructed to send to the learning data storage unit 922 belongs to the first processing 511 or the third processing. It can be determined from the processing 513. That is, when the node of the first process 511 is instructed, it can be determined that the image is divided into the layers 56 and when the node of the third process 513 is instructed, the image is divided into the layers 57. The band estimation result of the communication channel 23 by the band estimation unit 222 is passed to the third processing determination unit 1221.

  The third process determining unit 1221 divides a series of processes between the layers 56 when the estimated band is larger (wider) than the predetermined value, and otherwise between the layers 57. When the division is performed between the layers 56, the estimated band is passed to the dropout rate determination unit 12221 in the third processing learning unit 1222, and a sufficiently large band is transmitted to the dropout rate determination unit 2231 in the second processing learning unit 223. Pass as Note that a sufficiently large band means a band that satisfies Expression (5) in Expression (2) described above. It should be noted that any band may be used so long as p_dropout becomes p_dropout_best, and for example, may be infinite.

(Equation 5)
p_dropout_min ≤ p_dropout_best ... (5)

  The result of the forward propagation calculation of the third processing learning unit 1222 is passed to the second processing learning unit 223. Further, the network learning result by the second processing learning unit 223 is stored in the parameter storage unit 921 as a learned parameter, and is used for learning by the third processing learning unit 1222. This is set in the two processing unit 711.

  The third process learning unit 1222 performs the learning of the third process 513 in the same procedure as the second process learning unit 223, and stores the learned parameters in the parameter storage unit 921. Then, the learned parameter is set in the third processing unit 1211 via the parameter setting unit 713.

  When the third process determining unit 1221 divides the series of processes between the layers 57, the estimated band is passed to the dropout rate determining unit 2231 in the second process learning unit 223.

  The learning result of the network by the second processing learning unit 223 is stored in the parameter storage unit 921 as a learned parameter, and is set in the second processing unit 711 via the parameter setting unit 713.

  The learning data distribution unit 1223 passes the learning data stored in the learning data storage unit 922 to the second processing learning unit 223 or the third processing learning unit 1222, and instructs learning. If the learning data is that of the output node of the first process 511, the learning data is passed to the third process learning unit 1222, and the third process 513 and the second process 512 are transmitted together with the second process learning unit 223. Instruct students to learn. On the other hand, if the learning data is that of the output node of the third processing 513, the learning data is passed to the second learning unit 223, and an instruction is issued to perform the learning of the second processing 512.

  The teacher data correction instruction unit 911 transmits the teacher data to the teacher data correction unit 923 together with the time. The teacher data correction unit 923 associates the teacher data with the learning data of the time stored in the learning data storage unit 922, and passes the time to the learning data distribution unit 1223 to instruct re-learning.

<System operation>
FIG. 14 is a flowchart of the relearning process in the learning data distribution unit. This process is executed when the learning data distribution unit 1223 receives a re-learning instruction from the teacher data correction unit.

  In step S141, when the learning data distribution unit 1223 is instructed to re-learn from the teacher data correction unit 923, the learning data distribution unit 1223 acquires the timing Tt of the teacher data. In step S142, the learning data distribution unit 1223 deletes the learned parameters stored in the parameter storage unit 921 after the time Tt. This is because the learned parameters after that, including the teacher data that triggers the re-learning, are not needed to perform the re-learning.

  In step S143, the learning data distribution unit 1223 acquires the last learned parameter from the parameter storage unit 921. What is acquired here is the last learned parameter of each of the second process learning unit 223 and the third process learning unit 1222.

  In step S1431, the learning data distribution unit 1223 sets the learned parameter to each of the second processing learning unit 223 and the third processing learning unit 1222, and sets the initial state of the network. This is because the latest learned network at that time is set to the initial state in order to re-learn the mini-batch including the teacher data that triggers the re-learning.

  In step S144, the learning data distribution unit 1223 selects a mini-batch to be learned that includes the time Tt. If there is a mini-batch to be selected, the process proceeds to S1451, and if not, the process proceeds to S146.

  In step S1451, the learning data distribution unit 1223 determines whether the mini-batch learning data is the output of the first process 511. If it is the output of the first process 511, the series of processes is divided between the layers 56, and if not, the process is divided between the layers 57. If it is divided between the layers 56, the process of S1452 is performed, otherwise, the process of S1453 is performed.

  In step S1452, the learning data distribution unit 1223 instructs the third processing learning unit 1222 to perform learning using the selected mini-batch. At this time, as described above, the learning is performed in cooperation with the second processing learning unit 223. On the other hand, in step S1453, the learning data distribution unit 1223 instructs the second processing learning unit 223 to perform learning using the selected mini-batch. In this case, the learning by the third processing learning unit 1222 is not performed. In both cases of S1452 and S1453, at the end of learning, the learned parameters are stored in the parameter storage unit 921.

  In step S146, the learning data distribution unit 1223 selects a mini-batch next to the selected mini-batch. If there is a mini-batch, the process returns to S1451, and learning of the mini-batch is performed. If there is no mini-batch, that is, if learning of the last mini-batch has been completed, the process proceeds to S147.

  In step S147, the learning data distribution unit 1223 passes the learned parameter learned last to the parameter setting unit 713, and transmits the second processing network of the second processing unit 711 and the third processing unit 1211 of the third processing unit 1211. Update the network of the third process. Here, the process of S147 is performed at the end of the re-learning, but may be performed every time after S1452 and S1453.

  As described above, according to the eighth embodiment, learning can be efficiently performed even when the band of the communication path 23 changes with time.

(Other Examples)
The present invention supplies a program for realizing one or more functions of the above-described embodiments to a system or an apparatus via a network or a storage medium, and one or more processors in a computer of the system or the apparatus read and execute the program. This processing can be realized. Further, it can also be realized by a circuit (for example, an ASIC) that realizes one or more functions.

  2 System; 21 First case; 22 Second case; 23 Communication path; 211 Communication unit; 221 Communication unit; 214 First processing unit; 215 Data transfer control unit; 222 Band estimation unit; Learning unit: 2231 Dropout rate determination unit

Claims (10)

An image processing system using an L-layer neural network (NN),
First processing means for performing a first process using the NN of the preceding M layer of the L layer;
A second processing unit that performs the second processing using the NN of the subsequent (LM) layer of the L layer;
A parameter including a coupling coefficient from each node of the output layer of the first processing unit to each node of the input layer of the second processing unit based on an output value from the output layer of the first processing unit. Learning means to learn,
A drop-out node for suppressing transmission of an output value from each node in an output layer of the first processing means to the learning means in accordance with a communication band between the first processing means and the learning means Determining means for determining
Has,
The image processing apparatus according to claim 1, wherein the first processing unit transmits to the learning unit an output value from each node of the output layer of the first processing unit excluding the dropout node determined by the determining unit. system. The determining means determines a dropout rate in an output layer of the first processing means according to a communication band between the first processing means and the learning means, and determines the dropout rate based on the dropout rate. The image processing system according to claim 1, wherein a dropout node is determined. The data amount transmitted from the first processing unit to the learning unit is smaller than any data amount transmitted between the M layers of the NN used in the first processing unit. The image processing system according to claim 1. 2. The method according to claim 1, wherein the determining unit determines the dropout node such that the number of the dropout nodes with respect to the total number of nodes in the output layer of the first processing unit is equal to or greater than the dropout rate. 3. The image processing system according to 2. The NN of the M layer in the first processing means performs a convolution filter operation,
5. The image processing system according to claim 1, wherein the NN of the (LM) layer in the second processing unit performs a full connection operation. 6. The first processing means is disposed in a first device;
The learning means is arranged in a second device different from the first device,
The image processing system according to claim 1, wherein the first device and the second device are communicably connected to each other via a communication network. The second processing means is disposed in the first device,
7. The image processing system according to claim 6, wherein the first device further includes a setting unit that sets a parameter learned by the learning unit to each node of an input layer of the second processing unit. . The first device further includes providing means for providing teacher data indicating a correct processing result by the second processing means,
The second device comprises:
First storage means for storing output values transmitted from each node of the output layer of the first processing means;
A second storage unit for storing a learning result obtained by the learning unit;
Instruction means for correcting teacher data related to the output value stored in the first storage means based on the teacher data provided by the providing means, and instructing the learning means to re-learn;
The image processing system according to claim 6, further comprising: 9. The image processing system according to claim 8, wherein a holding period of the learning result in the second storage unit is longer than a holding period of the output value in the first storage unit. The image processing system according to claim 1, further comprising a changing unit configured to change the value of M according to the communication band.

Images