TWI824796B - Method for classifying images, computer device and storage medium - Google Patents

Abstract

The present application relates to an image processing technology and provides a method for classifying images, a computer device and a storage medium. The method includes: generating a first classification model based on a pre-acquired training image and a pre-acquired classification network, generating a second classification model based on a plurality of test images and the first classification model, the second classification model including a flattening layer, a fully connected layer and a classification layer; generating an initial feature matrix based on an image to be classified and the flattening layer; in response that a dimension of the initial feature matrix is smaller than a dimension of an initial weight matrix in the fully connected layer, obtaining a target feature matrix by increasing the dimension of the initial feature matrix; obtaining a target weight matrix by rearranging a plurality of elements of the initial weight matrix, and generating a classification result according to the target weight matrix, the target feature matrix and the classification layer. The present application can improve the efficiency of images classification.

Description

Image classification methods, computer equipment and storage media

The present invention relates to the field of image processing, and in particular, to an image classification method, computer equipment and storage medium.

In the current image classification network, the huge amount of calculations in the fully connected layer will cause the calculation to take a long time, causing the image classification speed to be slow. Therefore, how to improve the speed of image classification while ensuring the classification accuracy? , has become a problem that needs to be solved currently.

In view of the above, it is necessary to provide an image classification method, computer equipment and storage medium that can solve the problem of slow image classification caused by the difficulty in reducing the huge amount of calculations in the fully connected layer.

This application provides an image classification method. The image classification method includes: obtaining a classification network, and obtaining images to be classified, training images and multiple test images, and classifying the classification network based on the training images. The first classification model is obtained through training, and the prediction accuracy rate of the first classification model for the plurality of test images is calculated. If the prediction accuracy rate is less than the preset value, the prediction accuracy rate of the first classification model is calculated from the plurality of test images. Select the target image to adjust the first classification model to obtain a second classification model, wherein the second classification model includes a flattening layer, a fully connected layer and a classification layer, and input the image to be classified into the In the second classification model, and obtain the initial feature matrix output from the flattening layer, if the initial The dimension of the feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer. The initial feature matrix is dimensionally increased to obtain the target feature matrix. The elements in the initial weight matrix are rearranged to obtain the target weight. matrix, generate a target vector according to the target weight matrix and the target feature matrix, input the target vector into the classification layer, and obtain the classification result of the image to be classified.

According to an optional embodiment of the present application, said training the classification network based on the training image to obtain the first classification model includes: calculating the loss value of the classification network based on the training image, based on the The classification network is adjusted according to the loss value until the loss value drops to the minimum and then the adjustment is stopped to obtain the first classification model.

其中，L ^sup 是指所述損失值，

是指第i張訓練圖像的損失值，2N是指所述訓練圖像的圖像總量，i是指所述訓練圖像中的第i張訓練圖像，y _i 是指所述第i張訓練圖像的標註類別，j是指與所述第i張訓練圖像的標註類別相同的訓練圖像中的第j張訓練圖像，y _j 是指所述第j張訓練圖像的標註類別，N _yi 是指與所述第i張訓練圖像的標註類別相同的訓練圖像的數量，∥ _i≠j 為第一指示函數，當且僅當i=j時取零，當i≠j時取1，∥ _yi=yj 為第二指示函數，當且僅當y _i =y _j 時取零，當y _i ≠y _j 時取1，∥ _i≠k 為第三指示函數，當且僅當i=k時取零，當i≠k時取1，z _i 是指將所述第i張訓練圖像輸入到所述分類網路中得到的單位向量，z _j 是指將第j張訓練圖像輸入到所述分類網路中得到的單位向量，k是指除所述第i張訓練圖像之外的任意一張訓練圖像，z _k 是指將所述第k張訓練圖像輸入到所述分類網路中得到的單位向量，τ為預設的標量調節參數。 According to an optional embodiment of the present application, calculating the loss value of the classification network based on the training image includes: the calculation method of the loss value is:

Where, L ^sup refers to the loss value,

refers to the loss value of the i- th training image, 2 N refers to the total number of images in the training image, i refers to the i - th training image in the training image, y _i refers to the The annotation category of the i -th training image, j refers to the j- th training image among the training images with the same annotation category as the i -th training image, y _j refers to the j -th training image The annotation category of the image, N _yi refers to the number of training images with the same annotation category as the i- th training image, ∥ _{i ≠ j} is the first indicator function, which takes zero if and only if i = j , Takes 1 when i ≠ j , ∥ _{yi = yj} is the second indicator function, takes 0 if and only if y _i = y _j , takes 1 when y _i ≠ y _j , ∥ _{i ≠ k} is the third indicator function , takes zero if and only if i = k , takes 1 when i ≠ k , z _i refers to the unit vector obtained by inputting the i -th training image into the classification network, z _j refers to The unit vector obtained by inputting the j -th training image into the classification network, k refers to any training image except the i -th training image, z _k refers to the unit vector obtained by inputting the j-th training image into the classification network. The k training images are input into the unit vector obtained by the classification network, and τ is a preset scalar adjustment parameter.

According to an optional embodiment of the present application, calculating the prediction accuracy of the first classification model for the plurality of test images includes: obtaining the category labels of the plurality of test images, and converting the plurality of test images into The test image is input into the first classification model, the prediction result of each test image is obtained, the prediction result is compared with the corresponding category label, and the prediction result that is the same as the corresponding category label is determined To determine the target result, determine the test image corresponding to the target result as the target image, count the first number of the target images and the second number of the test images, and calculate the The ratio of the first quantity to the second quantity is the prediction accuracy rate.

According to an optional embodiment of the present application, selecting a target image from the plurality of test images to adjust the first classification model to obtain the second classification model includes: aggregating the plurality of target images. Class processing, generate multiple clustering centers, input the target image corresponding to each clustering center into the first classification model for training until the prediction accuracy rate is greater than or equal to the preset value, and obtain the Describe the second classification model.

According to an optional embodiment of the present application, if the dimension of the initial feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer, then performing dimensionality enhancement processing on the initial feature matrix to obtain the target feature matrix includes: Count the number of matrix rows and matrix columns of the initial feature matrix, multiply the number of matrix rows and the number of matrix columns to obtain a target product, and decompose the target product into prime factors to obtain multiple prime factors. Factor, combine any two identical prime factors among the multiple prime factors into a prime factor pair, and calculate the product of the two prime factors in the prime factor pair to obtain the characteristic product. Each prime factor can only be combined once , select a target prime factor pair from the prime factor pair according to the target product and the characteristic product, extract a prime factor in the target prime factor pair, and obtain a characteristic prime factor. According to the target prime factor pair The logarithm and the characteristic prime factors generate a characteristic value, the target prime factor pair is replaced with zero among the plurality of prime factors, and after the replacement is completed, all prime factors that are not zero are multiplied to obtain the target value. , based on the configuration value, the target value and the feature value, perform dimensionality enhancement processing on the initial feature matrix to obtain the target feature matrix.

According to an optional embodiment of the present application, rearranging the elements of the initial weight matrix to obtain the target weight matrix includes: determining the last element of each column in the initial weight matrix as the target element, and repeatedly The element at the head of the column is adjusted to the end of each column, and the positions of other elements in each column are moved to the head of the column in sequence until the target element moves to the head of the column and then stops. After permutation, the target weight matrix is obtained.

According to an optional embodiment of the present application, generating a target vector based on the target weight matrix and the target feature matrix includes: performing a multiplication operation on the target weight matrix and the target feature matrix to obtain the target vector.

The present application provides a computer device, which includes: a storage that stores at least one instruction; and a processor that executes the at least one instruction to implement the image classification method.

The present application provides a computer-readable storage medium. The computer-readable storage medium stores at least one instruction. The at least one instruction is executed by a processor in a computer device to implement the image classification method.

It can be seen from the above technical solutions that this application calculates the prediction accuracy rate of the first classification model for the plurality of test images, and then compares the prediction accuracy rate with the preset value to determine whether to Adjusting the first classification model can improve the prediction ability of the second classification model. Since only the target image is selected to adjust the first classification model during the adjustment process, the calculation time during the adjustment process can be reduced. quantity, thereby improving the generation efficiency of the second classification model, input the image to be classified into the second classification model, and obtain the initial feature matrix output from the flattening layer, if the dimensions of the initial feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer, then the initial feature matrix is dimensioned to obtain a target feature matrix. By changing the dimension of the target feature matrix, the amount of parameters in each operation can be increased, Obtain the initial weight matrix of the fully connected layer, rearrange the elements of the initial weight matrix to obtain the target weight matrix, perform a multiplication and accumulation operation on the target weight matrix and the target feature matrix to obtain the target vector, And input the target vector into the classification layer to obtain the classification result of the image to be classified. Since the elements in the initial weight matrix are rearranged and the order of operations of the elements is changed, it can ensure that each During each operation, elements in the weight matrix will participate in the operation, thereby avoiding the output of blank results (that is, avoiding invalid operations), thereby reducing the number of operations and the time spent in operations, thus speeding up image classification.

1:Computer equipment

2: Shooting equipment

12:Storage

13: Processor

101~109: Steps

Figure 1 is an application environment diagram of a preferred embodiment of the image classification method of the present application.

Figure 2 is a flow chart of a preferred embodiment of the image classification method of the present application.

Figure 3 is a schematic structural diagram of a computer device for implementing an image classification method according to a preferred embodiment of the present application.

In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

As shown in Figure 1, it is an application environment diagram of a preferred embodiment of an image classification method in this application. The image classification method can be applied to one or more computer devices 1. The computer device 1 communicates with the shooting device 2. The shooting device 2 can be a camera or other equipment that implements shooting, for example, The target object can be photographed by the photographing device 2 to obtain an image to be classified, where the target object can be an animal such as a cat or a dog, or a product such as a water cup or a toy.

The computer device 1 is a device that can automatically perform parameter value calculation and/or information processing according to preset or stored instructions. Its hardware includes, but is not limited to: microprocessor, application specific integrated circuit (Application Specific Integrated Circuit). Circuit (ASIC), Field-Programmable Gate Array (FPGA), Digital Signal Processor (DSP), embedded devices, etc.

The computer device 1 can be any computer product that can perform human-computer interaction with the user, such as a personal computer, a tablet computer, a smart phone, a personal digital assistant (Personal Digital Assistant, PDA), a game console, and an interactive Internet TV. (Internet Protocol Television, IPTV), wearable smart devices, etc. The computer equipment 1 may also include network equipment and/or user equipment. The network equipment includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

The network where the computer device 1 is located includes but is not limited to the Internet, wide area network, metropolitan area network, regional network, virtual private network (Virtual Private Network, VPN), etc.

As shown in Figure 2, it is a flow chart of a preferred embodiment of an image classification method of the present application. According to different needs, the order of each step in this flow chart can be adjusted according to the actual detection requirements, and some steps can be omitted. The execution subject of the method is a computer device, such as the computer device 1 shown in Figure 1 .

Step 101: Obtain the classification network, and obtain the images to be classified, training images and multiple test images.

In at least one embodiment of the present application, the classification network refers to a network that can predict the category of the image to be classified. In at least one embodiment of the present application, the image to be classified refers to an image that needs to be classified. In at least one embodiment of the present application, the training images may be used to train the classification network. In at least one embodiment of the present application, the multiple test images refer to images whose categories are known, and the multiple test images can be used to calculate the accuracy of the classification network. In at least one embodiment of the present application, the computer device constructs the classification network based on a convolutional neural network. Among them, the convolutional neural network can be a VGG16 network, a ResNet network, etc.

In at least one embodiment of the present application, the computer device acquiring the image to be classified includes: the computer device controlling the photographing device to capture the object to be classified to obtain the image to be classified. The objects to be classified may be animals such as cats and dogs, or plants such as flowers.

In at least one embodiment of the present application, the computer device obtains the training image from a preset first database. In at least one embodiment of the present application, the computer device obtains the plurality of test images and the category label of each test image from a preset second database. Wherein, the category label refers to the real category of the test image. The second database may be CIFAR-10, ImageNet or other databases.

In at least one embodiment of the present application, in order to ensure that the brightness and size of the training image and the multiple test images are more uniform, the acquired training image and the multiple test images can be equalized and Normalization processing. The training image and the multiple test images may include Images of multiple types of items such as animals (for example, puppies, kittens, etc.), plants (for example, flowers, trees, etc.).

Step 102: Train the classification network based on the training images to obtain a first classification model.

In at least one embodiment of the present application, the first classification model refers to a model obtained by training the classification network using the training images.

In at least one embodiment of the present application, the computer device trains the classification network based on the training image, and obtaining the first classification model includes: the computer device calculates the classification based on the training image. The loss value of the network. Further, the computer device adjusts the classification network based on the loss value until the loss value drops to a minimum and then stops adjusting, thereby obtaining the first classification model.

Through the above implementation, the classification is trained based on the loss value until the classification network converges, and the first classification model is obtained, which can improve the classification accuracy of the first classification model.

Specifically, the computer device calculating the loss value of the classification network based on the training image includes: the computer device inputs the training image into the classification network, and obtains the loss value of the training image. Label category, the computer device calculates the loss value based on the training image and the label category.

其中，L ^sup 是指所述損失值，

是指第i張訓練圖像的損失值，2N是指所述訓練圖像的圖像總量，i是指所述訓練圖像中的第i張訓練圖像，y _i 是指所述第i張訓練圖像的標註類別，j是指與所述第i張訓練圖像的標註類別相同的訓練圖像中的第j張訓練圖像，y _j 是指所述第j張訓練圖像的標註類別，N _yi 是指與所述第i張訓練圖像的標註類別相同的訓練圖像的數量，∥ _i≠j 為第一 指示函數，當且僅當i=j時取零，當i≠j時取1，∥ _yi=yj 為第二指示函數，當且僅當y _i =y _j 時取零，當y _i ≠y _j 時取1，∥ _i≠k 為第三指示函數，當且僅當i=k時取零，當i≠k時取1，z _i 是指將所述第i張訓練圖像輸入到所述分類網路中得到的單位向量，z _j 是指將第j張訓練圖像輸入到所述分類網路中得到的單位向量，k是指除所述第i張訓練圖像之外的任意一張訓練圖像，z _k 是指將所述第k張訓練圖像輸入到所述分類網路中得到的單位向量，τ為預設的標量調節參數。 The calculation method of the loss value is:

Where, L ^sup refers to the loss value,

refers to the loss value of the i- th training image, 2 N refers to the total number of images in the training image, i refers to the i - th training image in the training image, y _i refers to the The annotation category of the i -th training image, j refers to the j- th training image among the training images with the same annotation category as the i -th training image, y _j refers to the j -th training image The annotation category of the image, N _yi refers to the number of training images with the same annotation category as the i- th training image, ∥ _{i ≠ j} is the first indicator function, which takes zero if and only if i = j , Takes 1 when i ≠ j , ∥ _{yi = yj} is the second indicator function, takes 0 if and only if y _i = y _j , takes 1 when y _i ≠ y _j , ∥ _{i ≠ k} is the third indicator function , takes zero if and only if i = k , takes 1 when i ≠ k , z _i refers to the unit vector obtained by inputting the i -th training image into the classification network, z _j refers to The unit vector obtained by inputting the j -th training image into the classification network, k refers to any training image except the i -th training image, z _k refers to the unit vector obtained by inputting the i-th training image. The k training images are input into the unit vector obtained by the classification network, and τ is a preset scalar adjustment parameter.

Step 103: Calculate the prediction accuracy rate of the first classification model for the plurality of test images.

In at least one embodiment of the present application, the prediction accuracy rate refers to the probability that the first classification model correctly classifies the multiple test images.

In at least one embodiment of the present application, the computer device calculating the prediction accuracy of the first classification model for the multiple test images includes: the computer device obtains category labels of the multiple test images. , input the plurality of test images into the first classification model to obtain the prediction result of each test image, and the computer device compares the prediction result with the corresponding category label, and compares the prediction result with the corresponding category label. The prediction result with the same corresponding category label is determined as the target result. Further, the computer device determines the test image corresponding to the target result as the target image. Furthermore, the computer device counts multiple images. The first number of target images and the second number of the plurality of test images. Furthermore, the computer device calculates the ratio of the first number to the second number to obtain the The prediction accuracy rate.

Through the above implementation, the first number of target images and the second number of the plurality of test images are counted, and the proportion of the first number in the second number is calculated, which can quickly calculate Get the prediction accuracy of the first classification model.

Step 104: If the prediction accuracy rate is less than the preset value, select a target image from the plurality of test images to adjust the first classification model to obtain a second classification model, wherein the second classification model The model includes flattening layer, fully connected layer and classification layer.

In at least one embodiment of the present application, the target image refers to the first category A part of the test images corresponding to when the model accurately predicts the multiple test images.

In at least one embodiment of the present application, the computer device selects a target image from the plurality of test images to adjust the first classification model. Obtaining the second classification model includes: the computer device selects a target image from the plurality of test images and adjusts the first classification model. The target image is subjected to clustering processing to generate multiple clustering centers. Further, the computer device inputs the target image corresponding to each clustering center into the first classification model for training. If the prediction accuracy rate is greater than or equal to the preset value, the second classification model is obtained.

Among them, this embodiment performs clustering processing on multiple target images based on a K means clustering algorithm to generate multiple clustering centers. The preset value can be set by oneself and is not limited in this application.

Through the above implementation, a clustering algorithm is used to select the target image and adjust the first classification model until the prediction accuracy of the first classification model is greater than the preset value, and the second classification model is obtained. The classification model can improve the prediction accuracy of the second classification model. Since the target images are all correctly classified test images, adjustment using the target image can improve the classification of the second classification model. ability.

In other embodiments of the present application, if any target image is included in the clustering center, then any target image is input into the first classification model for training. If the clustering center contains If no target image is included, the target image closest to the cluster center is input into the first classification model for training. In other embodiments of the present application, after adjusting the first classification model using the target image corresponding to each cluster center, when the adjusted first classification model cannot be greater than or equal to the preset value , it is necessary to re-cluster the multiple target images to generate multiple new clustering centers to retrain the first classification model.

Step 105: Input the image to be classified into the second classification model, and obtain the initial feature matrix output from the flattening layer.

In at least one embodiment of the present application, the initial feature matrix refers to a matrix obtained after a dimension reduction operation by the flattening layer.

In at least one embodiment of the present application, the second classification model includes a convolutional layer, The computer device inputs the image to be classified into the second classification model, and obtaining the initial feature matrix output from the flattening layer includes: the computer device inputs the image to be classified into the second classification model. Feature extraction is performed in the convolution layer to obtain a target matrix, multiple dimensional values of the target matrix are multiplied to obtain the operation result, the value 1 is used as the number of matrix rows, and the operation result is used as the number of matrix columns, Based on the number of matrix rows and the number of matrix columns, the target matrix is transformed into the initial feature matrix, and the dimension of the initial feature matrix is two-dimensional.

Wherein, the dimension of the target matrix is greater than or equal to 3. The generation process of the initial feature matrix is basically the same as the inverse generation process of the target feature matrix below, and will not be described in detail here.

Through the above implementation, the high-dimensional target matrix output from the convolution layer is subjected to dimensionality reduction processing to obtain the initial feature matrix. Through the dimensionality reduction operation, the dimension size of the initial feature matrix can be controlled.

Step 106: If the dimension of the initial feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer, perform dimensionality enhancement processing on the initial feature matrix to obtain a target feature matrix.

In at least one embodiment of the present application, the target feature matrix refers to a matrix whose dimensions are consistent with those of the initial weight matrix.

In at least one embodiment of the present application, the computer device performs dimensionality enhancement processing on the initial feature matrix, and obtaining the target feature matrix includes: the computer device counts the number of matrix rows of the initial feature matrix and the The number of matrix columns, further, the computer device multiplies the matrix row number and the matrix column number to obtain a target product, and performs prime factor decomposition on the target product to obtain multiple prime factors, and further Specifically, the computer device combines any two identical prime factors among the plurality of prime factors into a pair of prime factors, and calculates the product of the two prime factors in the pair of prime factors to obtain a characteristic product, each prime factor It can only be combined once. The computer device selects a target prime factor pair from the prime factor pair according to the target product and the characteristic product. Further, the computer device extracts one prime factor from the target prime factor pair. factor to obtain a characteristic prime factor, the computer device generates a characteristic value according to the logarithm of the target prime factor pair and the characteristic prime factor, the computer device replaces the target prime factor pair among the plurality of prime factors is zero, after completing the replacement, all non-zero The prime factors are multiplied together to obtain the target value. Based on the configuration value, the target value and the feature value, the computer device performs a dimension-raising process on the initial feature matrix to obtain the target feature matrix.

Wherein, the target prime factor pair refers to a prime factor pair whose characteristic product can be divided by the target product, and the configuration value is a value of 1. It can be understood that the target product is large enough and non-prime to ensure that there must be at least two identical prime factors among the plurality of prime factors.

Specifically, the computer device generating a characteristic value based on the logarithm of the target prime factor pair and the characteristic prime factor includes: if the number of the target prime factor pair is single, the computer device determines the characteristic prime factor. is the characteristic value. If the number of target prime factor pairs is multiple, the computer device multiplies the characteristic prime factors to obtain the characteristic value.

Specifically, the computer device performs dimensionality enhancement processing on the initial feature matrix based on the configuration value, the target value and the feature value. Obtaining the target feature matrix includes: the computer device uses the configuration value as For the batch size, use the target value as the number of channels and the feature value as the number of rows and columns.

For example: the number of matrix rows of the initial feature matrix is 1, and the number of matrix columns of the initial feature matrix is 60, that is: the initial feature matrix is: [1 1 1 1 1 3 3 3 3 3 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3 1 1 1 1 1 1 1 1 1 1 3 3 3 3 3 2 2 2 2 2].

The configuration value is 1. The target value calculated through the above method is 15. The characteristic value is 2. Based on the configuration value 1 and the target value 15, the characteristic value 2 is the initial characteristic matrix. Perform dimensionality enhancement processing to obtain the target feature matrix, which is:

The target feature matrix includes a three-dimensional matrix, that is, the three-dimensional matrix includes 15 two-dimensional matrices, and the number of rows and columns of each two-dimensional matrix is both 2. In this embodiment, when all When the dimensions of the initial weight matrix in the connection layer are four dimensions, by uniformly converting the dimensions of the initial feature matrix into four dimensions, it can be ensured that the dimensions of the target feature matrix input to the fully connected layer are consistent with the initial weight matrix.

Through the above implementation, the dimensions of the initial feature matrix are converted to be consistent with the dimensions of the initial weight matrix, so that the target feature matrix can be directly multiplied with the initial weight matrix, so that the target feature matrix can be Multiple two-dimensional matrices are added to the operation, and the amount of parameters in each operation is increased, so the operation speed of the fully connected layer can be improved.

Step 107: Rearrange the elements in the initial weight matrix to obtain a target weight matrix.

In at least one embodiment of the present application, the target weight matrix refers to a matrix after elements are rearranged.

In at least one embodiment of the present application, the computer device rearranges the elements in the initial weight matrix to obtain the target weight matrix, including: the computer device rearranges the last element of each column in the initial weight matrix Determined as the target element, further, the computer device repeatedly adjusts the element at the head position of each column to the end position of each column, and moves the positions of other elements in each column to the head position in sequence until the target element Stop the arrangement after moving to the head position of the column, and obtain the target weight matrix.

Through the above implementation, all elements in the initial weight matrix can be rearranged, so that the order of operations of the elements in the initial weight matrix is more conducive to fast multiplication operations with the target feature matrix.

Step 108: Generate a target vector according to the target weight matrix and the target feature matrix.

In at least one embodiment of the present application, the target vector refers to the multiplication result of the target weight matrix and the target feature matrix.

In at least one embodiment of the present application, the computer device generating a target vector according to the target weight matrix and the target feature matrix includes: the computer device converts the target weight matrix into The matrix is multiplied by the target feature matrix to obtain the target vector.

Through the above implementation, since the operation order of the elements in the target weight matrix is changed, it can be ensured that the elements in the target feature matrix have corresponding elements in the target weight matrix during each operation. Operating with it improves the effectiveness of the operation and reduces the operation time.

Step 109: Input the target vector into the classification layer to obtain the classification result of the image to be classified.

In at least one embodiment of the present application, the classification result refers to the predicted category of the image to be classified by the second classification model. The classification results may include categories such as cats and dogs.

In at least one embodiment of the present application, the computer device inputs the target vector into the classification layer, and obtaining the classification result of the image to be classified includes: the computer device inputs the target vector into In the classification layer, the probability of each category is obtained, and the computer device uses the category corresponding to the probability with the largest value as the classification result. Wherein, the classification layer may be a softmax classifier.

By determining the category corresponding to the probability with the largest value as the classification result, the accuracy of the classification result can be improved.

It can be seen from the above technical solutions that this application calculates the prediction accuracy rate of the first classification model for the plurality of test images, and then compares the prediction accuracy rate with the preset value to determine whether to Adjusting the first classification model can improve the prediction ability of the second classification model. Since only the target image is selected to adjust the first classification model during the adjustment process, the calculation time during the adjustment process can be reduced. quantity, thereby improving the generation efficiency of the second classification model, input the image to be classified into the second classification model, and obtain the initial feature matrix output from the flattening layer, if the dimensions of the initial feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer, then the initial feature matrix is dimensioned to obtain a target feature matrix. By changing the dimension of the target feature matrix, the amount of parameters in each operation can be increased, Obtain the initial weight matrix of the fully connected layer, and rearrange the elements of the initial weight matrix to obtain the target weight matrix, and put the target weight matrix into The weight matrix and the target feature matrix are multiplied and accumulated to obtain a target vector, and the target vector is input into the classification layer to obtain the classification result of the image to be classified. Since the initial weight matrix is The elements in are rearranged and the order of operations of the elements is changed. Therefore, it is ensured that the elements in the weight matrix will participate in the operation in each operation, thereby avoiding the output of blank results (that is, avoiding invalid operations), thus reducing the cost of operations. The number of times and the time used for operations can therefore speed up image classification.

As shown in Figure 3, it is a schematic structural diagram of a computer device for implementing an image classification method according to a preferred embodiment of the present application.

In one embodiment of the present application, the computer device 1 includes, but is not limited to, a storage 12, a processor 13, and a computer program stored in the storage 12 and capable of running on the processor 13. , such as image classification programs. Those skilled in the art can understand that the schematic diagram is only an example of the computer device 1 and does not constitute a limitation on the computer device 1. It may include more or less components than shown in the diagram, or some components may be combined, or different components may be used. Components, for example, the computer device 1 may also include input and output devices, network access devices, buses, etc.

The processor 13 may be a central processing unit (CPU), or other general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC). , Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete component gate circuits or transistor components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can be any conventional processor, etc. The processor 13 is the computing core and control center of the computer device 1 and uses various interfaces and lines to connect the entire computer device 1 various parts, and obtain the operating system of the computer device 1 and various installed applications, program codes, etc. For example, the processor 13 can obtain the image to be classified captured by the shooting device 2 through an interface.

The processor 13 obtains the operating system of the computer device 1 and various installed applications. The processor 13 obtains the application program to implement each of the above image classification methods. The steps in the example are as shown in Figure 2.

For example, the computer program may be divided into one or more modules/units, and the one or more modules/units are stored in the storage 12 and retrieved by the processor 13, to complete this application. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions. The instruction segments are used to describe the acquisition process of the computer program in the computer device 1 .

The storage 12 can be used to store the computer programs and/or modules. The processor 13 runs or obtains the computer programs and/or modules stored in the storage 12 and calls the computer programs and/or modules stored in the storage. The information in 12 realizes various functions of the computer device 1. The storage 12 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; a storage data area. Areas can store information created based on the use of computer equipment, etc. In addition, the storage 12 may include non-volatile storage, such as a hard drive, memory, plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card , a memory card (Flash Card), at least one disk storage device, memory device, or other non-volatile solid-state storage device.

The storage 12 may be an external storage and/or an internal storage of the computer device 1 . Furthermore, the storage 12 may be a storage in a physical form, such as a memory stick, a TF card (Trans-flash Card), and so on.

If the integrated modules/units of the computer equipment 1 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above embodiment methods by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. The computer program can be stored in a computer-readable storage medium. When acquired by the processor, the computer program can implement the steps of each of the above method embodiments.

Wherein, the computer program includes computer program code, and the computer program code can be in the form of original program code, object code form, obtainable file or some intermediate form, etc. The computer-readable medium may include: any entity or device, recording medium, or device capable of carrying the computer program code. Pen drives, mobile hard drives, magnetic disks, optical disks, computer storage, read-only memory (ROM, Read-Only Memory).

2 , the storage 12 in the computer device 1 stores multiple instructions to implement an image classification method, and the processor 13 can obtain the multiple instructions to achieve: obtain a classification network, and obtain Images to be classified, training images and multiple test images; training the classification network based on the training images to obtain a first classification model; calculating the effect of the first classification model on the multiple test images The prediction accuracy rate of the image; if the prediction accuracy rate is less than the preset value, select a target image from the plurality of test images to adjust the first classification model to obtain a second classification model, wherein, the The second classification model includes a flattening layer, a fully connected layer and a classification layer; input the image to be classified into the second classification model, and obtain the initial feature matrix output from the flattening layer; if The dimension of the initial feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer. The initial feature matrix is dimensionally increased to obtain the target feature matrix; the elements in the initial weight matrix are rearranged to obtain the target weight matrix; generate a target vector according to the target weight matrix and the target feature matrix; input the target vector into the classification layer to obtain the classification result of the image to be classified.

Specifically, for the specific implementation method of the above instructions by the processor 13, reference can be made to the description of the relevant steps in the corresponding embodiment in Figure 2, which will not be described again here.

In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple networks. on the unit. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application can be integrated into one processing In a unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of hardware plus software function modules.

Therefore, the embodiments should be regarded as illustrative and non-restrictive from any point of view, and the scope of the present application is defined by the appended claims rather than the above description, and it is therefore intended that those falling within the claims All changes within the meaning and scope of the equivalent elements are included in this application. Any associated association markup in a request item should not be considered to limit the request item in question.

Furthermore, it is clear that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. Multiple units or devices stated in this application may also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names and do not indicate any specific order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified. Modifications or equivalent substitutions may be made without departing from the spirit and scope of the technical solution of the present application.

101~109: Steps

Claims (9)

An image classification method, applied to computer equipment, wherein the image classification method includes: obtaining a classification network, and obtaining images to be classified, training images and multiple test images; based on the training images The classification network is trained to obtain a first classification model; the prediction accuracy rate of the first classification model for the plurality of test images is calculated; if the prediction accuracy rate is less than a preset value, the prediction accuracy rate of the first classification model is calculated from the plurality of test images. Select the target image from the test image to adjust the first classification model to obtain a second classification model, where the second classification model includes a flattening layer, a fully connected layer and a classification layer; the image to be classified is The image is input into the second classification model, and the initial feature matrix output from the flattening layer is obtained; if the dimension of the initial feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer, the The initial feature matrix is dimensioned up to obtain a target feature matrix; the elements in the initial weight matrix are rearranged to obtain a target weight matrix; a target vector is generated according to the target weight matrix and the target feature matrix, including: The target weight matrix is multiplied by the target feature matrix to obtain the target vector; the target vector is input into the classification layer to obtain the classification result of the image to be classified. The image classification method according to claim 1, wherein said training the classification network based on the training image to obtain the first classification model includes: calculating the classification network based on the training image the loss value; adjust the classification network based on the loss value, stop adjusting until the loss value drops to the minimum, and obtain the first classification model. The image classification method according to claim 2, wherein calculating the loss value of the classification network based on the training image includes: the calculation method of the loss value is:

Where, L ^sup refers to the loss value,

refers to the loss value of the i- th training image, 2 N refers to the total number of images in the training image, i refers to the i - th training image in the training image, y _i refers to the The annotation category of the i -th training image, j refers to the j- th training image among the training images with the same annotation category as the i -th training image, y _j refers to the j -th training image The annotation category of the image, N _yi refers to the number of training images with the same annotation category as the i- th training image, ∥ _{i ≠ j} is the first indicator function, which takes zero if and only if i = j , Takes 1 when i ≠ j , ∥ _{yi = yj} is the second indicator function, takes 0 if and only if y _i = y _j , takes 1 when y _i ≠ y _j , ∥ _{i ≠ k} is the third indicator function , takes zero if and only if i = k , takes 1 when i ≠ k , z _i refers to the unit vector obtained by inputting the i -th training image into the classification network, z _j refers to The unit vector obtained by inputting the j -th training image into the classification network, k refers to any training image except the i -th training image, z _k refers to the unit vector obtained by inputting the i-th training image. The k training images are input into the unit vector obtained by the classification network, and τ is a preset scalar adjustment parameter. The image classification method according to claim 1, wherein calculating the prediction accuracy of the first classification model for the plurality of test images includes: obtaining category labels of the plurality of test images; The multiple test images are input into the first classification model to obtain the prediction result of each test image; the prediction result is compared with the corresponding category label, and the same as the corresponding category label The prediction result is determined as the target result; the test image corresponding to the target result is determined as the target image; the first number of the multiple target images and the second number of the multiple test images are counted. ; Calculate the ratio of the first quantity to the second quantity to obtain the prediction accuracy rate. The image classification method as described in claim 1, wherein the plurality of images are measured from Selecting a target image from the image to adjust the first classification model to obtain the second classification model includes: performing clustering processing on multiple target images to generate multiple cluster centers; The target image corresponding to the center is input into the first classification model for training until the prediction accuracy rate is greater than or equal to the preset value, and the second classification model is obtained. The image classification method as described in claim 1, wherein if the dimension of the initial feature matrix is smaller than the dimension of the initial weight matrix in the fully connected layer, the initial feature matrix is dimensionally upgraded, Obtaining the target feature matrix includes: counting the number of matrix rows and matrix columns of the initial feature matrix; performing a multiplication operation on the number of matrix rows and the number of matrix columns to obtain a target product; performing a prime factorization of the target product Decompose to obtain multiple prime factors; combine any two identical prime factors among the multiple prime factors into a pair of prime factors, and calculate the product of the two prime factors in the pair of prime factors to obtain the characteristic product, each The prime factors can only be combined once; select a target prime factor pair from the prime factor pair according to the target product and the characteristic product; extract one prime factor from the target prime factor pair to obtain a characteristic prime factor; according to the The logarithm of the target prime factor pair and the characteristic prime factor generate a characteristic value; replace the target prime factor pair with zero among the plurality of prime factors, and after completing the replacement, compare all prime factors that are not zero. Multiplication operation is performed to obtain the target value; based on the configuration value, the target value and the feature value, the initial feature matrix is dimensionally elevated to obtain the target feature matrix. The image classification method according to claim 1, wherein rearranging the elements of the initial weight matrix to obtain the target weight matrix includes: determining the last element of each column in the initial weight matrix as the target element; repeatedly adjust the element at the head of each column to the end of each column, and move the positions of other elements in each column to the head of the column in sequence, until the target element moves to the head of the column and stops arranging, we get The target weight matrix. A computer device, wherein the computer device includes: A memory that stores at least one instruction; and a processor that executes the at least one instruction to implement the image classification method as described in any one of claims 1 to 7. A computer-readable storage medium, wherein: at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is executed by a processor in a computer device to implement any one of claims 1 to 7 image classification method.

Images