JP6209879B2 - Convolutional neural network classifier system, training method, classification method and use thereof - Google Patents

Description

  The present invention relates to a classifier, specifically, a convolutional neural network (hereinafter abbreviated as CNN) classifier system, its use, a training method for a convolutional neural network classifier system, and a convolutional neural network classification. The present invention relates to a method of classifying an object using a container system and its use.

  So far, recognition of off-line handwritten characters is a difficult task. In recent years, identification methods based on convolutional neural networks have realized the development of current technology. However, the recognition rate of the current identification method is limited.

  In order to solve this technical problem, an identification method or classification method with a high recognition rate and high efficiency is required.

  One of the main objects of the present invention is a convolutional neural network classifier system and its use, a method for training a convolutional neural network classifier system, a method for classifying an object using a convolutional neural network classifier system, and its To provide a use.

  The following briefly describes the present invention and provides a basic understanding of the present invention. This brief description is not exhaustive for the invention. Further, there is no intention to determine the essential or important part of the present invention, and there is no intention to limit the scope of the present invention. It is understood that it is only that.

  According to one aspect of the invention, a convolutional neural network classifier system is provided. The system has M types of convolutional neural network classifiers that are dissimilar to each other, where M is an integer greater than 1, of which each type of convolutional neural network classifier includes a plurality of convolutional neural networks of the same type. Includes a classifier.

  According to another aspect of the invention, a method for training the convolutional neural network classifier system is provided. The method uses training samples to train all levels of convolutional neural network classifiers in the cascade structure, and for multiple convolutional neural network classifiers in each level of convolutional neural network classifiers. Randomly transforming each original training sample; classifying the original training sample using a first level convolutional neural network classifier; training difficult to classify with a first level convolutional neural network classifier Training the second-level convolutional neural network classifier as a training sample for the entire second-level convolutional neural network classifier; and the second-level convolutional neural network classifier to the M-th level convolution Repeating the above steps sequentially for any two adjacent levels up to the embedded neural network classifier.

  According to another aspect of the present invention, a method for classifying an object using the convolutional neural network classifier system is provided. This method classifies objects using a first level convolutional neural network classifier; and uses a second level convolutional neural network classifier to classify for the first level convolutional neural network classifier. Classifying difficult objects; and repeating the above steps sequentially for all two adjacent levels from the second level convolutional neural network classifier to the Mth level convolutional neural network classifier. .

  According to another aspect of the present invention, there is provided an application in which the convolutional neural network classifier system is used as an identification number.

  According to another aspect of the present invention, an application using the classification method as an identification number is provided.

  According to another aspect of the present invention, a computer program for realizing the above method is provided.

  According to another aspect of the invention, a computer program product in at least a computer readable medium format is provided, in which computer program code for implementing the method is stored.

  These and other advantages of the invention will become more apparent with reference to the following description of the invention with reference to the drawings.

The above and other objects, features and advantages of the present invention will become more apparent with reference to the following description of the invention with reference to the drawings. The components in the drawings merely illustrate the principles of the invention. In the drawings, the same or corresponding technical features or components are denoted by the same or corresponding reference numerals.
1 is a block diagram illustrating a convolutional neural network classifier system 100 according to one embodiment of the present invention. FIG. 3 is a block diagram illustrating a convolutional neural network classifier system 200 according to another embodiment of the present invention. Schematic showing a spatial linear sampling convolutional neural network classifier; It is a figure which shows the operation method of linear sampling. It is the schematic which shows the spatial linear sampling type | mold convolution neural network classifier which abbreviate | omitted the connection between some layers at random. It is the schematic which shows a spatial nonlinear sampling type convolution neural network classifier. It is the schematic which shows the operation process of a spatial nonlinear sampling. It is the schematic which shows the spatial linear sampling type | mold convolution neural network classifier which abbreviate | omitted the connection between some layers at random. FIG. 3 is a schematic diagram illustrating a convolutional neural network classifier without a spatial sampling layer. It is the schematic which shows the convolution neural network classifier which abbreviate | omits the connection between some layers at random and does not have a spatial sampling layer. 5 is a flowchart illustrating a training method 1100 for a convolutional neural network classifier system. 2 is a flow chart illustrating a method 1200 for classifying objects utilizing a convolutional neural network classifier system. It is a block block diagram of the example of the calculation equipment which can be used for implementation of this invention.

  Embodiments of the present invention will be described below with reference to the drawings. Elements and technical features described in one drawing or embodiment of the invention may be combined with elements and technical features in one or more other drawings or embodiments. It should also be noted that for the sake of clarity, the drawings and description omit the display and description of elements and processes known to those skilled in the art that are not relevant to the present invention.

  FIG. 1 is a block diagram illustrating a convolutional neural network classifier system 100 according to an embodiment of the present invention.

  As shown in FIG. 1, the convolutional neural network classifier system 100 includes M types of convolutional neural network classifiers that are different from each other, that is, from the first type of convolutional neural network classifier T1 to the Mth type of convolutional neural network. It has a network classifier TM. Here, M is an integer greater than 1.

  Each type of convolutional neural network classifier has a plurality of convolutional neural network classifiers of the same type. As shown in FIG. 1, the first type of convolutional neural network classifier T1 includes three convolutional neural network classifiers CNN-1, and the second type of convolutional neural network classifier T2 includes three convolutional neural networks. The classifier CNN-2 and the Mth type of convolutional neural network classifier TM each have three convolutional neural network classifiers CNN-M.

  In this example, each type of convolutional neural network classifier is shown to have three convolutional neural network classifiers of the same type, but each type of convolutional neural network classifier actually has a convolutional neural network. The number of classifiers is not limited to three, and may be an arbitrary number and may have a different number.

  FIG. 2 is a block diagram illustrating a convolutional neural network classifier system 200 according to an embodiment of the present invention.

  As shown in FIG. 2, the convolutional neural network classifier system 200 includes three types of convolutional neural network classifiers, each of which includes a first type of convolutional neural network classifier T1 and a second type of convolutional neural network classification. A classifier T2 and a third type of convolutional neural network classifier T3. The three types of convolutional neural network classifiers are different from each other and have a three-level cascade structure.

  As shown in FIG. 2, the first level convolutional neural network classifier T1 can classify objects to be classified. The second level convolutional neural network classifier T2 can classify objects that are difficult to classify for the first level convolutional neural network classifier T1. The third level convolutional neural network classifier T3 can classify objects that are difficult to classify for the second level convolutional neural network classifier T2. Uses all levels of convolutional neural network classifiers to target objects that are difficult to classify for all levels of convolutional neural network classifiers (here, objects that are difficult to classify for third level convolutional neural network classifiers) Voting and categorizing objects. For example, “difficult to classify” as used in this specification represents misclassification or misidentification of an object to be classified, or means that classification or identification of an object to be classified is correct but reliability is low. Can be represented.

  Each level of the convolutional neural network classifier can have a plurality of convolutional neural network classifiers of the same type. For example, a plurality of the same type of convolutional neural network classifiers in each level of the convolutional neural network classifier can classify objects through voting.

  Also, FIG. 2 shows a three level cascade structure, which is merely exemplary. Indeed, the convolutional neural network classifier system 200 may be any cascade structure of two or more levels. If the convolutional neural network classifier system 200 has an M level cascade structure, when classifying an object, all of the items from the first level convolutional neural network classifier to the Mth level convolutional neural network classifier are used. The above operations are sequentially performed on two adjacent levels, that is, the current level of the convolutional neural network classifier classifies the object to be classified, and the next level of the convolutional neural network classifier Further classify objects that are difficult to classify at the level of.

  In addition, the first type of convolutional neural network classifier T1, the second type of convolutional neural network classifier T2, and the third type of convolutional neural network classifier T3, which are different from each other, are fully utilized to efficiently and accurately target objects. Can be constructed based on the attributes of different types of convolutional neural network classifiers. For example, a convolutional neural network classifier for each level of the cascade structure can be constructed in the following manner: a convolutional neural network classifier that has a uniform effect on each type of sample is placed at the previous level in the cascade structure; Place convolutional neural network classifiers with prominent effects on specific types of samples at intermediate levels in the cascade structure. Also, sparse continuous broad convolutional neural network classifiers can be placed at a later level in the cascade structure.

  The convolutional neural network classifiers at each level in the cascade structure of the present invention can be selected from and are not limited to the following classifiers: a) Spatial linear sampling convolutional neural network classifiers; b) Spatial linear sampling convolutional neural network classifier that omits connections between some layers randomly; c) Spatial nonlinear sampling convolutional neural network classifier; d) Connections between some layers randomly omitted Non-spatial linear sampling convolutional neural network classifier; e) convolutional neural network classifier without spatial sampling layer; and f) convolutional neural network classification without spatial sampling layer, omitting connections between some layers randomly vessel.

  Hereinafter, the six types of convolutional neural network classifiers that are different from each other will be described with reference to FIGS.

  FIG. 3 is a schematic diagram showing a spatial linear sampling type convolutional neural network classifier. FIG. 4 is a diagram illustrating a linear sampling operation method.

  As shown in FIG. 3, a typical structure of a spatial linear sampling convolution neural network classifier ("FM" is a feature diagram) consists of an input layer, alternating convolution layers and linear sampling layers, Finally, the output layer is included. Here, the number of times the convolutional layer and the linear sampling layer are alternately arranged is unlimited.

  As shown in FIG. 4, the operation method of linear sampling is to obtain the product sum of each component of one 2 × 2 block and each component of one linear window function having the same size (actually large size). Is not limited to 2x2.) In FIG. 3, regarding the feature diagrams at both ends of the connection line, the feature diagram on the right side of the connection line is obtained by convolution or linear sampling of the feature diagram on the left side of the connection line.

Hereinafter, the convolution operation will be described. For example, in FIG. 3, when there is a connecting line between the input image and the feature diagram FM11 and this connecting line represents a convolution operation, this connecting line itself becomes one template, and the calculation formula of FM11 is
FM11 = Input image * A template on a connecting line.

  Here, “*” represents a convolution operation.

Also, for example, in FIG. 3, when FM 31 has a connecting line between FM 21 and FM 22, and these two connecting lines represent a convolution operation, each of these two connecting lines itself becomes one template. Correspondingly, the calculation formula of FM32 is
FM31 = FM21 * Template on connecting line + FM22 * Template on connecting line.

  Here, “*” represents a convolution operation.

Hereinafter, the linear sampling operation will be described. In the linear sampling operation method, as shown in FIG. 4, FM31 and FM41 in FIG. 4 correspond to FM31 and FM41 in FIG. The product sum of each component of one 2 × 2 pixel block in the feature diagram FM31 and one component of one linear window function having the same size is obtained, and the result is given to the corresponding pixel in the feature diagram FM41. The formula is
S _i = P _i · W _i (i = 1, 2, 3, 4).

  When actually used, the size of the pixel block and the linear window function are not limited to 2 × 2, and the linear window function can be arbitrarily set.

  FIG. 5 is a schematic diagram showing a spatial linear sampling type convolutional neural network classifier in which connection between some layers is omitted at random.

  Compared with the spatial linear sampling type convolutional neural network classifier shown in FIG. 3, the structure of the convolutional neural network classifier shown in FIG. 5 has a connection between some feature diagrams (for example, between FM21 and FM31). Was randomly removed. That is, spatial linear sampling type convolutional neural network classifier + sparse random connection.

  FIG. 6 is a schematic diagram showing a spatial nonlinear sampling type convolutional neural network classifier.

  Compared with the spatial linear sampling type convolutional neural network classifier shown in FIG. 3, the spatial nonlinear sampling type convolutional neural network classifier shown in FIG. 6 employs a nonlinear sampling operation method. FIG. 7 is a schematic diagram showing an operation process of nonlinear sampling.

  In the spatial nonlinear sampling operation method, as shown in FIG. 7, FM31 and FM41 in FIG. 7 correspond to FM31 and FM41 in FIG. One 2 × 2 pixel block in the feature diagram FM31 is put into one nonlinear window function, and the result is given to the corresponding pixel in the feature diagram FM41. When actually used, the size of the pixel block and the linear window function are not limited to 2 × 2, and the setting of the nonlinear window function is also arbitrary. Nonlinear window functions that are relatively common include those that take a maximum value, a minimum value, or an intermediate value.

  FIG. 8 is a schematic diagram showing a spatial nonlinear sampling convolutional neural network classifier (ie, spatial nonlinear sampling convolutional neural network classifier + sparse random connection) in which connections between some layers are randomly omitted.

  Compared to the spatial nonlinear sampling type convolutional neural network classifier shown in FIG. 5, a partial characteristic diagram of the spatial nonlinear sampling type convolutional neural network classifier shown in FIG. 8 in which connection between some layers is randomly omitted. The connection between, for example, the connection between FM21 and FM31 was randomly removed.

  FIG. 9 is a schematic diagram illustrating a convolutional neural network classifier without a spatial sampling layer.

  Compared to the previous four types of convolutional neural network classifiers, the convolutional neural network classifier without a spatial sampling layer shown in FIG. 9 does not have a spatial linear sampling layer and a non-spatial linear sampling layer.

  FIG. 10 is a schematic diagram illustrating a convolutional neural network classifier without a spatial sampling layer (ie, a convolutional neural network classifier without a spatial sampling layer + sparse random connection) in which connections between some layers are randomly omitted. It is.

  Compared to the convolutional neural network classifier without the spatial sampling layer shown in FIG. 9, the typical structure of the convolutional neural network classifier without the spatial sampling layer shown in FIG. Connections between feature diagrams, such as the connection between FM31 and FM51, were randomly removed.

  In one embodiment of the present invention, the convolutional neural network classifier system 100 or 200 includes a reliability calculation classifier (FIGS. 1 and 2) that calculates a reliability when the convolutional neural network classifier classifies an object. (Not shown).

  When actually using a convolutional neural network classifier, a sample to be identified is given, and the convolutional neural network classifier needs to output a reliability corresponding to the predicted number. For each convolutional neural network classifier at each level in the cascade structure, classify the training sample using the convolutional neural network classifier, and for each training sample from at least some nodes in each layer from the convolutional neural network classifier Extract the response values and train a confidence calculation classifier based on the response values. In this way, not only information on a specific layer but also information on each layer of the convolutional neural network classifier can be fully utilized.

  Specifically, for a convolutional neural network classifier, the process of training a reliability calculation classifier is as follows: give a training sample, send it to the convolutional neural network classifier, perform forward propagation, and Response values of at least some neuron nodes are extracted, and a reliability calculation classifier is trained using the response values. In the actual identification process, the current identification object is sent to the convolutional neural network classifier and forward propagation is performed to obtain the classification result of the convolutional neural network classifier. At the same time, response values of the at least some neuron nodes are extracted, and reliability is calculated using a trained reliability calculation classifier based on the response values.

  Hereinafter, an example in which the reliability calculation classifier is a support vector machine will be described.

  Given training samples, feed them into a convolutional neural network classifier, perform forward propagation, extract response values of at least some neuron nodes in each layer, and construct eigenvectors (and record sample categories). That is, each training sample constitutes one eigenvector. All eigenvectors are sent to a support vector machine (SVM) and trained to obtain support vectors for each category. These support vectors are used for the reliability calculation. Specifically, in the actual identification process, the current sample to be identified is sent to the convolutional neural network classifier to perform forward propagation, and then the classification result of the convolutional neural network classifier is acquired. At the same time, the selected neuron node response value is extracted to construct an eigenvector, the distance of the predicted number of the eigenvector to the category support vector is calculated, and the reciprocal of this distance is used as the reliability.

  The case where the reliability calculation classifier is a support vector machine has been described above. However, the reliability calculation classifier here may be any appropriate type of classifier and is not limited to a support vector machine. It is understood that there is no.

  FIG. 11 is a flowchart of a training method 1100 for a convolutional neural network classifier system.

  As shown in FIG. 11, in step S1102, training samples are used to train all levels of convolutional neural network classifiers in the cascade structure. Here, each of the convolutional neural network classifiers in each level of the convolutional neural network classifier is randomly transformed with respect to the original training sample. The training sample can be enriched by deforming it randomly. In this way, by training all the convolutional neural network classifiers using the original training sample after deformation, these convolutional neural network classifiers have a certain classification performance, and then training by level. Is advantageous.

  In step S1104, the initial value of the loop control variable N can be set to 1.

  In step S1106, the original training samples can be classified using an Nth level convolutional neural network classifier.

  In step S1108, the training samples that are difficult to classify by the Nth level convolutional neural network classifier are all training samples for training the N + 1th level convolutional neural network classifier, and the N + 1th level convolutional neural network is obtained. Train the classifier. Here, it is determined whether each convolutional neural network classifier is difficult to classify the training sample based on the reliability and / or the known mark of each training sample itself.

  In step S1110, the value of the loop control variable N is incremented by one.

  In step S1112, it is determined whether or not the loop control variable N is greater than or equal to M. M is the level of the cascade structure, where M ≧ 2.

  If N is smaller than M as a result of the determination in step S1112, the process returns to step S1116, and if N is M or more, the process ends.

  Also, several convolutional neural network classifiers can be trained before building the cascade structure. Each convolutional neural network classifier is one of the six types of heterogeneous structures. The training method directly adopts a typical stochastic gradient descent method. Train multiple times for each convolutional neural network classifier and use multiple samples each time. In the training process, the process in which each sample is sent to the convolutional neural network classifier is as follows: after one sample is given, the sample is randomly transformed (randomly rotated or moved), and then the transformed The sample is sent to the convolutional neural network classifier, forward propagation is performed, then backward propagation is performed by the stochastic gradient descent method, and the parameters of each layer of the convolutional neural network classifier are corrected in the backward propagation.

  Subsequently, a trained partial convolutional neural network classifier is selected based on certain rules to form a cascade classifier.

  Also, for each trained convolutional neural network classifier, using all training samples, the error rate in each type of sample in the current convolutional neural network classifier and in all samples in the current convolutional neural network classifier An average error rate is calculated, and all trained convolutional neural network classifiers are classified into the following three types based on the calculated numerical values: (1), sparse continuous convolutional neural network classifiers; (2) Except for (1) above, a convolutional neural network classifier having a low average error rate; and (3) a convolutional neural network classifier having a low error rate in a certain type of sample. Then, a part of the convolutional neural network classifiers are selected from the respective types of convolutional neural network classifiers and used for the cascade structure. Preferably, the cascade structure is constructed in the following manner. That is, the (2) class convolutional neural network classifier is arranged at the front level of the cascade structure, the (3) class convolutional neural network classifier is arranged at the intermediate level of the cascade structure, and the (1) Arrange the class on the back level of the cascade structure.

  In addition, the reliability can be calculated using a reliability calculation classifier. First, train the reliability calculation classifier. Specifically, for each convolutional neural network classifier at each level in the cascade structure, training of the sample classification is performed using the convolutional neural network classifier, and each training sample is subjected to each layer from the convolutional neural network classifier. Extract response values of at least some nodes, and train the reliability calculation classifier using the response values. Then, when the convolutional neural network classifier classifies the object, the reliability of the classifier is calculated using the reliability. Preferably, at least some of the nodes of each layer include at least some of the nodes of the second layer from the back.

  FIG. 12 is a flowchart of a method 1200 for classifying objects using a convolutional neural network classifier system.

  As shown in FIG. 12, in step S1202, the initial value of the loop control variable N is set to 1.

  In step S1204, an object is classified using an Nth level convolutional neural network classifier.

  In step S1206, the N + 1 level convolutional neural network classifier is used to classify objects that are difficult for the Nth level convolutional neural network classifier to classify.

  In step S1208, the value of the loop control variable N is incremented by one.

  In step S1210, it is determined whether N is M or more. Here, M is the number of levels of the cascade structure, and M ≧ 2.

  As a result of the determination in step S1210, if N is smaller than M, the process returns to step S1204. If N is M or more, the process is terminated.

  In addition, the reliability can be calculated using a reliability calculation classifier. For each classification target, each convolutional neural network classifier at each level in the cascade structure extracts the response value of the corresponding node when classifying each training sample; on the basis of the response value, a reliability calculation classifier is used. To calculate the reliability. The predetermined nodes include at least some nodes of the second layer from the back.

  Based on the reliability, each convolutional neural network classifier in each level determines whether it is difficult to classify the object.

  The convolutional neural network classifier system of the present invention can be used to identify numbers. Classification methods utilizing a convolutional neural network classifier system can also be used for number identification.

  Hereinafter, a process of identifying handwritten characters using a three-level cascaded convolutional neural network classifier system will be described. Here, reliability is also used.

  First, an identification object is given and it is sent to a first level convolutional neural network classifier. The level of the convolutional neural network classifier outputs two values, the predicted number and the corresponding reliability. If the reliability is greater than a predetermined threshold, the identification result is reliable, the identification result is output directly, and no subsequent steps are performed. Otherwise, the identification object is sent to the second level convolutional neural network classifier and the process for the first level convolutional neural network classifier is repeated. If the reliability is smaller than the predetermined threshold, the identification object is sent to the third level convolutional neural network classifier, and the process for the first level convolutional neural network classifier is repeated. If the reliability is still smaller than the predetermined threshold even after passing through the third level convolutional neural network classifier, the identification result of the identification object is obtained by voting of all levels of the convolutional neural network classifier.

  It should be noted here that each level of the convolutional neural network classifier can include a plurality of convolutional neural network classifiers of the same type. At this time, the output of each level is determined by voting of all the convolutional neural network classifiers in the level, and the reliability depends on the reliability of all the convolutional neural network classifiers in the level. For example, it is possible to take an average value of the reliability output from all the convolutional neural network classifiers in the level.

  According to the technique of the present invention, the identification accuracy for the international standard handwritten digit database MNIST is higher than that of some internationally famous methods, particularly the method using a single type of convolutional neural network classifier. Hereinafter, the reason why the technique of the present invention brings about a good technical effect will be described.

  First, the technique of the present invention uses the irrelevance between various types of heterogeneous convolutional neural network classifiers, and thus has improved the role of voting when raising the accuracy of identification. Voting by multiple classifiers can achieve a higher identification rate. However, as a precondition for realizing a higher identification rate, these classifiers are unrelated, for example, different types. Voting by a large number of classifiers of the same type cannot necessarily increase the identification rate significantly. The technique of the present invention satisfies this prerequisite because voting is performed by various kinds of heterogeneous convolutional neural network classifiers.

  Next, the technology of the present invention can fully utilize the pattern information provided by the convolutional neural network classifier, partly by conventional methods. In the technique of the present invention, a characteristic vector extracted from each layer of the convolutional neural network classifier is used to form an eigenvector, and thereby the reliability of the identification result of the convolutional neural network classifier is measured. The information provided by each layer of the convolutional neural network classifier can be fully utilized without relying only on the last layer of information of the classifier. From the viewpoint of pattern recognition theory, the neuron nodes in each layer in the convolutional neural network classifier provide constant pattern information, so if only the last layer information is used, the pattern information provided by each intermediate layer is It will be lost.

  Also, compared to existing handwritten digit identification methods that do not rely on a convolutional neural network classifier, the structure of the convolutional neural network classifier employed in the technology of the present invention has a large number of useful and previously unknown features, In contrast to being able to automatically learn from the input samples, existing methods require manually setting the features to be extracted in advance. Therefore, the number of features that can be extracted by existing methods is limited, and manpower is required at the feature setting stage.

  While the basic principles of the present invention have been described using specific embodiments, it should be emphasized that all or any of the steps or components of the method and apparatus of the present invention can be any computing device (processor or memory). It will be understood by those skilled in the art that hardware, firmware, software, or combinations thereof can be implemented in a network of computing devices (including media etc.) or computing devices. This can also be realized by those skilled in the art using the basic programming skills after reading the specification of the present invention.

  Therefore, the object of the present invention can be realized by executing one or a set of programs in an arbitrary computing device. The computing device may be a conventional general purpose device. The object of the present invention can be realized only by providing a program product including a program code for realizing the method or apparatus. That is, such a program product also constitutes the present invention, and a storage medium storing such a program product also constitutes the present invention. Of course, the storage medium may be any conventional storage medium or any storage medium developed in the future.

  When the embodiment of the present invention is realized through software and / or firmware, a program that configures the software is stored in a computer having a dedicated hardware structure through a storage medium or a network, for example, a general-purpose computer 1300 as shown in FIG. When installed, the computer can execute various functions when each type of program is installed.

  In FIG. 13, the central processing unit (CPU) 1301 performs various processes according to a program stored in a read-only memory (ROM) 1302 or a program uploaded from a storage unit 1308 to a random access memory (RAM) 1303. Execute. The RAM 1303 stores data necessary for the CPU 1301 to execute various processes as necessary. The CPU 1301, ROM 1302 and RAM 1303 are connected to each other via a bus 1304. An input / output interface 1305 is also connected to the bus 1304.

  The following elements are also connected to the input / output interface 1305: an input unit 1306 including a keyboard and a mouse; an output unit 1307 including a monitor and a speaker such as a cathode ray tube (CRT) and a liquid crystal display (LCD); A recording unit 1308 including a communication unit 1309 including a network interface card such as a LAN card and a modem. The communication unit 1309 performs communication processing via a network, for example, the Internet. A drive 1310 is also connected to the input / output interface 1305 as needed. A removable medium 1311 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor storage device or the like is inserted into the drive 1310 as necessary, and the computer program read from the medium is read into the recording unit 1308 as necessary. Installed.

  When the series of processing is realized through software, a program constituting the software is installed from a network, for example, the Internet, or a storage medium, for example, a removable medium 1311.

  As will be understood by those skilled in the art, the recording medium here is a removable medium 1311 in which the program is recorded and distributed separately from the equipment and provided to the user as shown in FIG. Not limited to. Examples of the removable medium 711 include a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including a read-only memory (CD-ROM) and a digital versatile disk (DVD) using a compact disk (CD)), This includes a magneto-optical disk (including a mini disk (MD) (registered trademark)) and a semiconductor memory device. The recording medium may be a hard disk included in the ROM 1302 or the recording unit 1308. The program is recorded in it and distributed to the user together with the equipment for recording it.

  The present invention provides a program product storing machine-readable instruction codes. The method of the embodiment of the present invention can be executed when the instruction code is read and executed by the machine.

  A storage medium of a program product storing the machine-readable instruction code is also included in the disclosure of the present invention. The storage medium includes a floppy disk, a magnetic disk, an optical disk, a memory card, a memory stick, and the like.

  It will be appreciated by those skilled in the art that the examples herein are illustrative and that the invention is not limited to these examples.

  The descriptions such as “first”, “second”, and “Nth” in the present specification are for distinguishing the related features in terms of letters and describing the present invention more clearly. Therefore, it has no limiting meaning.

  By way of example, each step of the method and each component module and / or unit of the facility can be implemented by software, firmware, hardware, or a combination thereof, and become part of the facility. Means or methods that can be used when each component module or unit of the facility is combined by software, firmware, hardware, or a combination thereof are well known to those skilled in the art, and thus the description thereof is omitted here.

  For example, when the present invention is realized through software or firmware, a program constituting the software is installed from a storage medium or a network to a computer having a dedicated hardware structure (for example, the general-purpose computer 1300 shown in FIG. 13). Can do. The computer can realize various functions when various programs are installed.

  Features described and / or illustrated for one embodiment may be used in one or more other embodiments in the same or similar manner, and may be combined or otherwise combined with features in other embodiments. The features of this embodiment can be substituted.

  It should be emphasized that the term “comprising / comprising” herein refers to the presence of a feature, element, step or module and does not exclude the presence or addition of one or more other features, elements, steps or modules. .

  Further, the method of the present invention is not limited to the time order described in the specification, and may be executed in another time order, in parallel, or independently. Therefore, the technical scope of the present invention is not limited to the execution order of the method described in the specification.

  It should be appreciated that the present invention can be designed with various modifications, improvements or equivalents to the present invention within the spirit and scope of the appended claims. In addition, the scope of the present invention is not limited to the specific embodiments of the processes, equipment, means, methods and steps described in the specification. Those skilled in the art will understand that existing or future-developed processes, equipment, means, methods or methods having basically the same functions and basically the same effects as the embodiments of the present invention from the contents disclosed in the present invention. Steps can be used or executed. Accordingly, the spirit of the appended claims includes such processes, facilities, means, methods, or steps.

The following supplementary notes are further disclosed with respect to the embodiments including the above examples.
(Appendix 1)
A convolutional neural network classifier system,
M types of convolutional neural network classifiers that are heterogeneous to each other, where M is an integer greater than 1,
Each type of convolutional neural network classifier comprises a plurality of convolutional neural network classifiers of the same type,
A convolutional neural network classifier system characterized by that.
(Appendix 2)
The convolutional neural network classifier system according to appendix 1, wherein the M kinds of convolutional neural network classifiers different from each other are arranged in an M-level cascade structure.
(Appendix 3)
The first level convolutional neural network classifier is used to classify objects,
The second level convolutional neural network classifier is used to classify objects that are difficult to classify for the first level convolutional neural network classifier,
Repeating the operation sequentially for all two adjacent levels from the second level convolutional neural network classifier to the Mth level convolutional neural network classifier;
The convolutional neural network classifier system according to appendix 2.
(Appendix 4)
The plurality of the same type of convolutional neural network classifiers in each level of the convolutional neural network classifier classifies the object through voting.
The convolutional neural network classifier system according to appendix 2.
(Appendix 5)
A convolutional neural network classifier with a consistent effect on each type of sample is placed at the front level in the cascade structure,
A convolutional neural network classifier configuration that has a prominent effect on a particular type of sample is placed at an intermediate level in the cascade structure,
Placing a convolutional neural network classifier at each level in the cascade structure;
The convolutional neural network classifier system according to any one of appendices 2 to 4.
(Appendix 6)
A sparse continuous broad convolutional neural network classifier is placed at a back level in the cascade structure;
The convolutional neural network classifier system according to appendix 5.
(Appendix 7)
Each level of the convolutional neural network classifier in the cascade structure is:
a) a spatial linear sampling convolutional neural network classifier;
b) a spatial linear sampling convolutional neural network classifier in which the connections between some layers are omitted randomly;
c) a spatially nonlinear sampling convolutional neural network classifier;
d) a spatially nonlinear sampling convolutional neural network classifier in which connections between some layers are randomly omitted;
e) a convolutional neural network classifier without a spatial sampling layer;
f) Convolutional neural network classifiers with no spatial sampling layer, the connection between some layers is randomly omitted,
A convolutional neural network classifier system according to any one of supplementary notes 1 to 6 selected from:
(Appendix 8)
A reliability calculation classifier that calculates a reliability when the convolutional neural network classifier classifies the object;
For each convolutional neural network classifier in each level in the cascade structure, train the sample classification using the convolutional neural network classifier, and for each training sample from each layer of the convolutional neural network classifier Extract response values for at least some nodes,
The convolutional neural network classifier system according to any one of appendices 1 to 7, which trains the reliability calculation classifier based on the response value.
(Appendix 9)
Any one of appendices 3 to 8, wherein the convolutional neural network classifiers of all levels use voting for objects that are difficult to classify, so that the classification of the objects is determined. A convolutional neural network classifier system according to claim 1.
(Appendix 10)
A method of performing training using the convolutional neural network classifier system according to any one of appendices 2 to 9,
The training samples are used to train all levels of the convolutional neural network classifier in the cascade structure, and the original training samples are applied to each of the convolutional neural network classifiers in each level of the convolutional neural network classifier. Randomly deform,
Classifying the original training sample using the first level convolutional neural network classifier;
Training samples that are difficult to classify for the first level convolutional neural network classifier are all training samples for training the second level convolutional neural network classifier, and train the second level convolutional neural network classifier And
Repeating the above steps sequentially for any two adjacent levels from the second level convolutional neural network classifier to the Mth level convolutional neural network classifier;
A method involving that.
(Appendix 11)
For each convolutional neural network classifier in each level in the cascade structure, training the sample classification using the convolutional neural network classifier, and for each training sample, at least one of each layer from the convolutional neural network classifier The response values of the nodes
The method according to claim 10, wherein a reliability calculation classifier is trained using the response value, and the reliability calculation classifier calculates a reliability when the convolutional neural network classifier classifies the object. .
(Appendix 12)
The method of appendix 11, wherein at least some of the nodes in each layer include at least some of the nodes in the second layer from the back.
(Appendix 13)
13. The method of any of clauses 10-12, wherein each convolutional neural network classifier determines whether it is difficult to classify the training sample based on confidence and / or a known mark of each training sample itself.
(Appendix 14)
A method for classifying an object using the convolutional neural network classifier system according to any one of appendices 1 to 9,
Classifying the object using the first level convolutional neural network classifier;
Using a second level convolutional neural network classifier to classify the object that is difficult to classify for the first level convolutional neural network classifier;
A method comprising sequentially repeating the above steps for all two adjacent levels from the second level convolutional neural network classifier to the Mth level convolutional neural network classifier.
(Appendix 15)
Using a convolutional neural network classifier at all levels to determine a classification of the objects by voting on objects that are difficult to classify at all levels of the convolutional neural network classifier;
The method according to appendix 14.
(Appendix 16)
For each classification object, extract the response value of the corresponding node when each convolutional neural network classifier at each level in the cascade structure classifies each training sample,
Based on the response value, the reliability is calculated using the reliability calculation classifier.
The method according to appendix 14.
(Appendix 17)
The method according to claim 16, wherein the predetermined nodes include at least some nodes of the second layer from the back.
(Appendix 18)
18. The method according to any one of appendices 10 to 17, wherein each convolutional neural network classifier in each level determines whether it is difficult to classify the object based on the reliability.
(Appendix 19)
Use of identifying numbers by using the convolutional neural network classifier system according to any one of appendices 1 to 9.
(Appendix 20)
Use for identifying numbers using the method according to any one of appendices 14 to 18.

Claims (7)

A convolutional neural network classifier system,
M types of convolutional neural network classifiers that are heterogeneous to each other, where M is an integer greater than 1,
Each type of convolutional neural network classifier comprises a plurality of convolutional neural network classifiers of the same type ,
The M types of convolutional neural network classifiers that are different from each other are arranged in an M-level cascade structure,
A convolutional neural network classifier with a consistent effect on each type of sample is placed at the front level in the cascade structure,
A convolutional neural network classifier that has a prominent effect on a particular type of sample is arranged in a convolutional neural network classifier at each level in the cascade structure such that a convolutional neural network classifier is arranged at an intermediate level in the cascade structure. Neural network classifier system. The first level convolutional neural network classifier is used for the first classification operation for classifying objects,
A second level convolutional neural network classifier is used for a second classification operation for classification of objects that are difficult to classify for the first level convolutional neural network classifier,
Sequentially repeating the first classification operation and the second classification operation for all the adjacent two levels from the second level of the convolutional neural network classifier to the M level convolutional neural network classifier to claim 1 Described convolutional neural network classifier system. A reliability calculation classifier that calculates a reliability when the convolutional neural network classifier classifies the object;
For each convolutional neural network classifier in each level in the cascade structure, the training sample is classified using the convolutional neural network classifier, and at least one from each layer of the convolutional neural network classifier for each training sample. The response values of the nodes
The convolutional neural network classifier system according to claim 2 , wherein the confidence calculation classifier is trained based on the response value. A method of training using a convolutional neural network classifier system according to claim 2 or 3 ,
Training levels are used to train all levels of convolutional neural network classifiers in the cascade structure, with each of the convolutional neural network classifiers in each level having original training samples. Randomly deform,
Classifying the original training sample using the first level convolutional neural network classifier;
Classification hard training samples for a convolutional neural network classifier of the first level, and all the training samples to train the second level of the convolutional neural network classifier, the second level of the convolutional neural network classifier Train and
Repeating the steps in sequence for any two adjacent levels from the second level convolutional neural network classifier to the Mth level convolutional neural network classifier. For each convolutional neural network classifier in each level in the cascade structure, the training sample is classified using the convolutional neural network classifier, and at least one of each layer from the convolutional neural network classifier for each training sample. The response values of the nodes
By using the response value train reliability calculation classifier, the reliability calculation classifier calculates the reliability when the convolutional neural network classifier for classifying the object, according to claim 4 the method of. A method for classifying an object using the convolutional neural network classifier system according to claim 2 , comprising:
Classifying the object using the first level convolutional neural network classifier;
Using the second level convolutional neural network classifier to classify the objects that are difficult to classify for the first level convolutional neural network classifier;
Method comprising sequentially repeating the above steps, it for all the adjacent two levels of the the second level of the convolutional neural network classifier to neural network classifier convolution of the first M level. 4. A convolutional neural network classifier system according to any one of claims 1 to 3 for identifying numbers.

Images