TWI769724B - Image feature extraction method and image feature extraction device, electronic device and storage media - Google Patents

Abstract

The present application provides an image feature extraction method and an image feature extraction device, an electronic device and a storage medium. The method includes: converting images to an image matrix, and dividing the image matrix to obtain image sub-blocks, dividing the weight matrix to obtain weight vectors, and dividing each weight vector to obtain weight sub-blocks, dividing offset matrix to obtain offset quantum blocks, multiplying the weight sub-blocks of the weight vector with the image sub-blocks to obtain the feature sub-blocks, adding the feature sub-blocks and the offset quantum blocks to obtain image feature sub-blocks, and the image feature sub-blocks making up image feature. The present application can reduce the amount of computation in the neural network in the process of image feature extraction, thus improving efficiency of image feature extraction.

Description

Image feature extraction method, device, electronic device and storage medium

The present application relates to the field of image processing, and in particular to an image feature extraction method, device, electronic device and storage medium.

Before classifying, identifying and detecting an image, it is usually necessary to extract the features of the image. The existing image feature extraction methods mainly use the method of directly multiplying the image and the weight vector to achieve feature extraction, so that the image Inefficiencies like feature extraction.

In view of the above content, it is necessary to propose an image feature extraction method, apparatus, electronic device and storage medium to improve the efficiency of image feature extraction.

A first aspect of the present application provides an image feature extraction method, the method includes: acquiring an image, converting the image into an image matrix, dividing the image matrix to obtain an image vector, and dividing the image vector to obtain a first number of image sub-blocks; obtain a weight matrix, divide the weight matrix to obtain a second number of weight vectors, divide each of the weight vectors to obtain a first number of weight sub-blocks; obtain an offset matrix, dividing the offset matrix to obtain a second number of offset quantum blocks; Perform vector multiplication on the first number of image sub-blocks and the first number of weight sub-blocks of each of the weight vectors to obtain a second number of feature sub-blocks, and combine the second number of feature sub-blocks with the first number of feature sub-blocks. Two numbers of offset quantum blocks are added to obtain image feature sub-blocks, and the image feature sub-blocks constitute image features.

Preferably, each of the image sub-block, the weight sub-block, and the offset sub-block has a preset number of rows and a preset number of columns, the preset number of columns and the preset number of rows same number.

Preferably, the dividing the image matrix to obtain the image vector includes: dividing the image matrix according to the preset number of rows to obtain a preliminary image vector set, where the preliminary image vector set includes at least one preliminary image vector set. image vector; determine whether there is a preliminary image vector with the number of rows less than the preset number of rows in the preliminary image vector set; when there is an image with the number of rows less than the preset number of rows in the preliminary image vector vector, use the preset value to fill the preliminary image vector with the number of rows less than the preset number of rows to obtain a preliminary image vector with the number of rows equal to the preset number of rows, and use the preliminary image vector as the image vector; or when there is no image vector with a row number smaller than the preset row number in the preliminary image vector, the preliminary image vector is used as the image vector.

Preferably, dividing the image vector to obtain the first number of image sub-blocks includes: dividing the image vector according to the preset number of columns to obtain the first number of preparatory image sub-blocks; determining the preparatory image sub-blocks Whether there is a preliminary image sub-block with a number of columns smaller than the preset number of columns in the image sub-block; when there is a preliminary image sub-block with a number of columns less than the preset number of columns in the preliminary image sub-block, Filling the preparatory image sub-blocks with a number of columns less than the preset number of columns using a preset value to obtain preparatory image sub-blocks with a number of columns equal to the preset number of columns, and using the first number of preparatory image sub-blocks as the first number of preparatory image sub-blocks a number of image sub-blocks; Or when there is no preliminary image sub-block with a column number smaller than the preset number of columns in the preliminary image sub-block, the first number of preliminary image sub-blocks is used as the first number of images subblock.

Preferably, dividing the weight matrix to obtain a second number of weight vectors includes: dividing the weight matrix according to the number of columns to obtain a second number of preparatory weight vectors; judging whether the preparatory weight vector has a number of columns less than The preset weight vector with the preset number of columns; when there is a weight vector with the number of columns less than the preset number of columns in the prepared weight vector, use the preset value to fill the number of columns less than the preset number of columns. obtain a preliminary weight vector with the number of columns equal to the preset number of columns, and use the second number of preliminary weight vectors as the second number of the weight vectors; or when there is no column number in the preliminary weight vector When the number of weight vectors is smaller than the preset number of columns, the second number of preparatory weight vectors is used as the second number of weight vectors.

Preferably, dividing each of the weight vectors to obtain a first number of weight sub-blocks includes: dividing each of the weight vectors according to the number of rows to obtain a fourth number of preparatory weight sub-blocks; determining the fourth number of weight sub-blocks; Whether it is equal to the first quantity, when the fourth quantity is smaller than the first quantity, calculate the first difference between the fourth quantity and the first quantity, and increase the reserve value according to the first difference The number of weight sub-blocks until the first number of pre-weight sub-blocks are obtained; or when the fourth number is greater than the first number, the first number of pre-weight sub-blocks is selected from the fourth number of pre-weight sub-blocks ; Judging whether there is a preparatory weight sub-block whose number of columns is less than the preset number of columns in the preparatory weight sub-blocks of the first quantity; When the number of columns is pre-weight sub-blocks, use a preset value to fill the pre-weight sub-blocks with a number of columns smaller than the preset number of columns to obtain pre-weight sub-blocks with a number of columns equal to the preset number of columns, and use the pre-weight sub-blocks of the first number sub-block as the first number of weights heavy sub-block; or when there is no pre-weight sub-block with a number of columns less than the preset number of columns in the first number of pre-weight sub-blocks, the first number of pre-weight sub-blocks is used as the first The number of weighted sub-blocks.

Preferably, the dividing the offset matrix to obtain the second number of offset sub-blocks includes: dividing the offset matrix according to the preset number of rows to obtain a preliminary offset vector, from which the preliminary offset vector is obtained. Select an offset vector from the offset vectors; divide the offset vector according to the preset number of columns to obtain a fifth number of preliminary offset sub-blocks; determine whether the fifth number is equal to the first Two quantities, when the fifth quantity is smaller than the second quantity, calculate a second difference between the fifth quantity and the second quantity, and increase the amount of the preparatory offset quantum block according to the second difference number until a second number of preparatory offset quantum blocks are obtained; or when the fifth number is greater than the second number, select a second number of preparatory offset quantum blocks from the fifth number of preparatory offset quantum blocks block; determine whether there is a preliminary offset quantum block with the number of columns less than the preset number of columns or the number of rows less than the preset number of rows in the second number of preliminary offset quantum blocks; when the second number of preliminary offset quantum blocks exist When there are preliminary offset sub-blocks with the number of columns smaller than the preset number of columns or the number of rows smaller than the preset number of rows in the shift sub-block, use the preset value to fill the number of columns smaller than the preset number of columns or the number of rows smaller than the preset number of rows. The number of rows of preliminary offset sub-blocks is obtained to obtain preliminary offset sub-blocks with the number of columns equal to the preset number of columns and the number of rows equal to the preset number of rows, and the second number of preliminary offset sub-blocks is used as the second number of offset sub-blocks. Shift sub-blocks; or when there is no pre-shift sub-block with a number of columns smaller than the preset number of columns or a number of rows smaller than the preset number of rows in the second number of preliminary offset sub-blocks, transfer the second number of The preliminary offset quantum blocks serve as the second number of offset quantum blocks.

A second aspect of the present application provides an image feature extraction device, the device comprising: An image sub-block acquisition module is used to acquire an image, convert the image into an image matrix, divide the image matrix to obtain an image vector, and divide the image vector to obtain a first number of image sub-blocks. block; a weight sub-block acquisition module is used to acquire a weight matrix, divide the weight matrix to obtain a second quantity of weight vectors, and divide each of the weight vectors to obtain a first quantity of weight sub-blocks; an offset quantum block acquisition module group, used to obtain an offset matrix, and divide the offset matrix to obtain a second number of offset sub-blocks; an image feature calculation module, used to compare the first number of image sub-blocks with each of the The first number of weight sub-blocks of the weight vector are vector multiplied to obtain a second number of feature sub-blocks, and the second number of feature sub-blocks and the second number of offset sub-blocks are added to obtain image feature sub-blocks block, the image feature sub-blocks constitute image features.

A third aspect of the present application provides an electronic device, the electronic device includes: a memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to implement the image feature extraction method.

A fourth aspect of the present application provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the image feature extraction method is implemented.

In this application, in this embodiment, by dividing the image matrix, the weight matrix and the deviation matrix into sub-blocks for operation, the calculation amount of the convolution operation in the neural network during the image feature extraction process is reduced, thereby improving the The efficiency of extracting features from images.

30: Image feature extraction device

301: Image sub-block acquisition module

302: Weight sub-block acquisition module

303: Offset quantum block acquisition module

304: Image feature calculation module

6: Electronic equipment

61: Memory

62: Processor

63: Computer Programs

S11~S14: Steps

FIG. 1 is a flowchart of an image feature extraction method in an embodiment of the present application.

FIG. 2 is a structural diagram of an image feature extraction apparatus according to an embodiment of the present application.

FIG. 3 is a schematic diagram of an electronic device in an embodiment of the present application.

FIG. 4 is a schematic diagram of an image feature extraction method in an embodiment of the present application.

In order to more clearly understand the above objects, features and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.

In the following description, many specific details are set forth in order to facilitate a full understanding of the present invention, and the described embodiments are only some, but not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein in the description of the present invention are for the purpose of describing specific embodiments only, and are not intended to limit the present invention.

Preferably, the image feature extraction method of the present application is applied in one or more electronic devices. The electronic device is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs) , Programmable Gate Array (Field-Programmable Gate Array, FPGA), Digital Signal Processor (Digital Signal Processor, DSP), embedded devices, etc.

The electronic device may be a computing device such as a desktop computer, a notebook computer, a tablet computer, and a cloud server. The electronic device can perform human-computer interaction with the user by means of a keyboard, a mouse, a remote control, a touch pad or a voice control device.

Example 1

FIG. 1 is a flowchart of an image feature extraction method in an embodiment of the present application. The image feature extraction method is applied in electronic equipment. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

Referring to FIG. 1 , the image feature extraction method specifically includes the following steps.

Step S11, acquiring an image, converting the image into an image matrix, dividing the image matrix to obtain an image vector, and dividing the image vector to obtain a first number of image sub-blocks.

In at least one embodiment of the present application, the converting the image into an image matrix includes: reading the image; acquiring a primitive point in the image; converting the three primary colors of the primitive point The encoded values form primitive vectors, and the primitive vectors form the image matrix.

In at least one embodiment of the present application, the dividing the image matrix to obtain the image vector includes: dividing the image matrix according to the preset number of rows to obtain a preliminary image vector set, the preliminary image vector set. The image vector set includes at least one preliminary image vector; it is judged whether there is a preliminary image vector whose number of rows is less than the preset number of rows in the preliminary image vector set; When the image vector with the preset number of lines is used, the preset value is used to fill the preliminary image vector with the number of lines less than the preset number of lines to obtain a preliminary image vector with the number of lines equal to the preset number of lines, and the The preparatory image vector is used as the image vector; or when there is no image vector whose number of rows is less than the preset number of rows in the preparatory image vector, the preparatory image vector is used as the image vector. like a vector.

In at least one embodiment of the present application, dividing the image vector to obtain the first number of image sub-blocks includes: dividing the image vector according to the preset number of columns to obtain the first number of preliminary images subblock; Judging whether there is a preparatory image subblock with a number of columns less than the preset number of columns in the preparatory image subblock; when there is a preparatory image in the preparatory image subblock with a number of columns smaller than the preset number When sub-blocks, use a preset value to fill the pre-image sub-blocks with the number of columns less than the preset number of columns to obtain pre-image sub-blocks with the number of columns equal to the preset number of columns, and fill the pre-image sub-blocks with the first number of pre-image sub-blocks. as the first number of image sub-blocks; or when there is no preliminary image sub-block with a number of columns less than the preset number of columns in the preliminary image sub-block, the first number of preliminary image sub-blocks image sub-blocks as the first number of image sub-blocks.

For example, the preset number of rows may be 3 rows, the preset number of columns may be 3 columns, and the preset value may be 0. When the number of rows of the image matrix is 11 and the number of columns is 11, divide the image matrix by 3 rows to obtain 4 preliminary image vectors, one of which is 2 rows and 11 columns, using Fill the preliminary image vector with 0 to obtain four image vectors of 3 rows and 11 columns; divide the image vector of 3 rows and 11 columns by 3 columns to obtain 4 preliminary image sub-blocks, one of which is a preliminary image sub-block For 3 rows and 2 columns, use a preset value of 0 to fill the preliminary image sub-block to obtain 4 image sub-blocks with 3 rows and 3 columns.

For another example, referring to FIG. 4 , the image vector A is divided according to the preset number of columns to obtain m preliminary image sub-blocks. When the number of columns of the m-th preliminary image sub-block is less than the preset number of columns, 0 is used. Fill the m-th preliminary image sub-block to obtain m image sub-blocks.

Step S12, obtaining a weight matrix, dividing the weight matrix to obtain a second number of weight vectors, and dividing each of the weight vectors to obtain a first number of weight sub-blocks.

In at least one embodiment of the present application, dividing the weight matrix to obtain the second quantity of weight vectors includes: dividing the weight matrix according to the number of columns to obtain the second quantity of preliminary weight vectors; judging the preliminary weights Whether there is a preliminary weight vector with the number of columns less than the preset number of columns in the vector; When there is a weight vector with a number of columns less than the preset number of columns in the prepared weight vector, use the preset value to fill the prepared weight vector with a number of columns less than the preset number of columns, so that the number of columns is equal to the Preliminary weight vectors with a preset number of columns, using the second number of preliminary weight vectors as the second number of weight vectors; or when there is no weight vector with a number of columns less than the preset number of columns in the preliminary weight vector , taking the second quantity of preparatory weight vectors as the second quantity of the weight vectors.

In at least one embodiment of the present application, dividing each of the weight vectors to obtain the first number of weight sub-blocks includes: dividing each of the weight vectors according to the number of rows to obtain a fourth number of preparatory weight sub-blocks ; Determine whether the fourth quantity is equal to the first quantity, and when the fourth quantity is less than the first quantity, calculate the first difference between the fourth quantity and the first quantity, and according to the The first difference value increases the number of preparatory weight sub-blocks until the first number of preparatory weight sub-blocks is obtained; or when the fourth number is greater than the first number, selects the first number of preparatory weight sub-blocks from the fourth number of sub-blocks. a number of pre-weight sub-blocks; determine whether there are pre-weight sub-blocks with a number of columns smaller than the preset number of columns in the first number of pre-weight sub-blocks; when there are pre-weight sub-blocks in the first number of pre-weight sub-blocks When the number of columns is less than the preset number of columns of the pre-weight sub-blocks, using a preset value to fill the pre-weight sub-blocks with the number of columns less than the preset number of columns to obtain the pre-weight sub-blocks of which the number of columns is equal to the preset number of columns, Take the first number of preparatory weight sub-blocks as the first number of weight sub-blocks; or when there is no preparatory weight sub-block with the number of columns less than the preset number of columns in the first number of preparatory weight sub-blocks, use The first number of preparatory weight sub-blocks is used as the first number of weight sub-blocks.

In at least one embodiment of the present application, the weight matrix is used to measure the importance of features in the image matrix.

In at least one embodiment of the present application, the weight matrix may be a matrix in an encoding layer of an autoencoder or a matrix in a convolutional layer of a convolutional neural network, and the weight matrix may be stored in In artificial intelligence chips, it is used to perform general matrix multiplication operations.

For example, when the weight matrix has 11 rows and 11 columns, divide the weight matrix by 3 rows to obtain 4 preliminary weight vectors, one of which is 2 rows and 11 columns, and is filled with 0 The preliminary weight vector, four weight vectors with 3 rows and 11 columns are obtained; the weight vector with 3 rows and 11 columns is divided by 3 columns to obtain 4 preliminary weight sub-blocks, one of which is a preliminary weight sub-block with 3 rows and 2 columns, using The preset weight sub-block is filled with a preset value of 0 to obtain four weight sub-blocks with 3 rows and 3 columns.

For another example, referring to FIG. 4 , divide the weight matrix B according to the preset number of columns to obtain n columns of preliminary weight vectors, and when the number of columns of the nth column of preliminary weight vectors is less than the preset number of columns, use 0 to fill the nth column. Prepare the weight vector to obtain n-column weight vector; divide each column of the n-column weight vector according to the preset number of rows to obtain m preliminary weight blocks, when the number of columns of the mth preliminary weight sub-block is less than the preset number of columns When , use 0 to fill the m-th preparatory weight sub-block to obtain m weight sub-blocks.

Step S13: Obtain an offset matrix, and divide the offset matrix to obtain a second number of offset sub-blocks.

In at least one embodiment of the present application, the dividing the offset matrix to obtain the second number of offset sub-blocks includes: dividing the offset matrix according to the preset number of rows to obtain preliminary offsets selecting an offset vector from the preliminary offset vectors; dividing the offset vector according to the preset number of columns to obtain a fifth number of preliminary offset sub-blocks; judging the first Whether the five quantity is equal to the second quantity, when the fifth quantity is smaller than the second quantity, calculate the second difference between the fifth quantity and the second quantity, and calculate the second difference according to the second difference Increase the number of preparatory offset quantum blocks until a second number of preparatory offset quantum blocks are obtained; or when the first When the number of five is greater than the second number, select a second number of preliminary offset quantum blocks from the fifth number of preliminary offset quantum blocks; determine whether there is a column in the second number of preliminary offset quantum blocks The number of preliminary offset sub-blocks is less than the preset number of columns or the number of rows is less than the preset number of rows; when the number of columns in the second number of preliminary offset sub-blocks is less than the preset number of columns or the number of rows is less than When there are pre-set offset sub-blocks with a preset number of rows, use a preset value to fill in the pre-offset sub-blocks whose number of columns is less than the preset number of columns or the number of rows is less than the preset number of rows, so that the number of columns is equal to the preset number of columns and the number of rows is equal to the number of preparatory offset quantum blocks, the second number of preparatory offset quantum blocks is used as the second number of preparatory offset quantum blocks; or when the second number of preparatory offset quantum blocks is When there is no preparatory offset quantum block whose number of columns is less than the preset number of columns or the number of rows is less than the preset number of rows, use the second number of the preparatory offset quantum blocks as the second number of offset quantum blocks .

In at least one embodiment of the present application, the offset matrix is used to measure the difficulty of generating positive excitation or negative excitation by the neural network.

In at least one embodiment of the present application, the offset matrix may be a matrix in an encoding layer of an autoencoder, or may be a matrix in a convolutional layer of a convolutional neural network, and the offset Matrices can be stored in AI chips for general-purpose matrix multiplication.

For example, when the number of rows of the weight matrix is 11 and the number of columns is 11, the weight matrix is divided into 3 rows to obtain 4 preliminary weight vectors, and the first preliminary weight vector is selected as the weight vector; Divide the weight vector of 3 rows and 11 columns to obtain 4 preparatory weight sub-blocks, one of which is 3 rows and 2 columns, and use the preset value 0 to fill the preparatory weight sub-block to obtain 4 pre-weight sub-blocks of 3 rows and 3 The weight subblock for the column.

For another example, referring to FIG. 4 , the offset vector C is divided according to the preset number of columns to obtain n preliminary offset sub-blocks, when the number of columns of the nth preliminary offset sub-block is less than the preset number of columns. , fill the nth preparatory offset quantum block with 0 to obtain n offset quantum blocks.

Step S14, performing vector multiplication on the first number of image sub-blocks and the first number of weight sub-blocks of each of the weight vectors to obtain a second number of feature sub-blocks, and combining the second number of feature sub-blocks with The second number of offset sub-blocks are added to obtain image feature sub-blocks, and the image feature sub-blocks constitute image features.

For example, using the 4 weight sub-blocks of the weight vector and the 4 image sub-blocks of the image vector are respectively multiplied and summed to obtain 4 feature sub-blocks, and the 4 feature sub-blocks are combined with the 4 offset sub-blocks Four image feature sub-blocks are obtained by adding, and the four image feature sub-blocks constitute image features.

For another example, as shown in FIG. 4 , a vector multiplication operation is performed on the image vector A and the weight vector in each column of the weight matrix, that is, the m image sub-blocks in the image vector A and the m weight sub-blocks in each column of the weight vector. The blocks are multiplied and summed in turn to obtain n feature sub-blocks, and the n feature sub-blocks and n offset sub-blocks are added in turn to obtain n image feature sub-blocks, and n image feature sub-blocks form a graph like features.

In at least one embodiment of the present application, in at least one embodiment of the present application, each of the image sub-block, the weight sub-block, and the offset sub-block has a preset number of rows equal to The preset number of columns, the preset number of columns is the same as the preset number of rows.

In at least one embodiment of the present application, the image feature extraction method can be used to determine similar images.

Specifically, using the image feature extraction method to determine similar images includes: acquiring a first image to be detected and a second image to be detected, and comparing the first image to be detected and the second image to be detected The image is converted into a first to-be-detected image vector and a second to-be-detected image vector; using the image feature extraction method to calculate the first to-be-detected image feature of the first to-be-detected image vector and the described The second to-be-detected image feature of the second to-be-detected image vector; using a preset error function to calculate an error function value between the first to-be-detected image feature and the second to-be-detected image feature; When the error function value is less than a preset threshold, it is determined that the first image to be detected and the second image to be detected are the same image, and when the error function value is greater than the preset threshold, it is determined that the first image to be detected and the second image to be detected are the same image The first image to be detected and the second image to be detected are different images.

In other embodiments of the present application, the image feature extraction method may also be used in an autoencoder.

Specifically, the image feature extraction method can be applied to the encoding layer and/or decoding layer of the self-encoder, and reconstructing the image by the encoder includes: acquiring the image to be reconstructed, and converting the image to be reconstructed. The reconstructed image is converted into an image vector to be reconstructed; the image vector to be reconstructed is input into the coding layer of the self-encoder, and the coding layer of the self-encoder uses the image feature extraction method to obtain reconstruction Image latent vector; input the reconstructed image latent vector into the decoding layer of the self-encoder, and the decoding layer of the self-encoder uses the image feature extraction method to obtain the reconstruction of the image to be reconstructed. compose the image.

In this embodiment, by dividing the image matrix, the weight matrix and the deviation matrix into sub-blocks for operation, the calculation amount of the convolution operation in the neural network in the process of image feature extraction is reduced, thereby improving the accuracy of the image in the image. The efficiency of feature extraction.

Example 2

FIG. 2 is a structural diagram of an image feature extraction apparatus 30 in an embodiment of the present application.

In some embodiments, the image feature extraction apparatus 30 runs in an electronic device. The image feature extraction device 30 may include a plurality of functional modules composed of program code segments. The program codes of each program segment in the image feature extraction device 30 can be stored in the memory and executed by at least one processor to perform the image feature extraction function.

In this embodiment, the image feature extraction apparatus 30 can be divided into a plurality of functional modules according to the functions performed by the image feature extraction apparatus 30 . Referring to FIG. 2, the image feature extraction device 30 may include an image sub-block acquisition module 301, a weight sub-block acquisition module 302, an offset sub-block acquisition module 303 and the Like feature calculation module 304 . A module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can perform fixed functions, and are stored in a memory. In some embodiments, the functions of each module will be described in detail in subsequent embodiments.

The image sub-block acquisition module 301 acquires an image, converts the image into an image matrix, divides the image matrix to obtain an image vector, and divides the image vector to obtain a first number of image sub-blocks. piece.

In at least one embodiment of the present application, converting the image into an image matrix by the image sub-block acquiring module 301 includes: reading the image; acquiring primitive points in the image; The three primary color-coded values of the primitive points are composed of primitive vectors, and the primitive vectors are composed of the image matrix.

In at least one embodiment of the present application, dividing the image matrix by the image sub-block obtaining module 301 to obtain an image vector includes: dividing the image matrix according to the preset number of rows to obtain a preliminary image Like a vector set, the preparatory image vector set includes at least one preparatory image vector; it is judged whether there is a preparatory image vector with a row number less than the preset row number in the preparatory image vector set; when the preparatory image vector set is When there is an image vector with the number of rows less than the preset number of rows in the vector, use the preset value to fill the preliminary image vector with the number of rows less than the preset number of rows to obtain a row number equal to the preset number of rows. number of preliminary image vectors, take the preliminary image vector as the image vector; or when there is no image vector with a row number less than the preset number of rows in the preliminary image vector, use the preliminary image vector as the image vector An image vector is prepared as the image vector.

In at least one embodiment of the present application, the image sub-block obtaining module 301 dividing the image vector to obtain the first number of image sub-blocks includes: dividing the image vector according to the preset number of columns Dividing to obtain a first number of preliminary image sub-blocks; judging whether there is a preliminary image sub-block whose number of columns is less than the predetermined number of columns in the preliminary image sub-block; when there is a preliminary image sub-block in the preliminary image sub-block When the number of columns is less than the preset number of columns of the preparatory image sub-block, use a preset value to fill the pre-image sub-block with the number of columns less than the preset number of columns to obtain a preparatory image with the number of columns equal to the preset number of columns sub-blocks, taking the first number of preliminary image sub-blocks as the first number of image sub-blocks; or when there is no preliminary image with a number of columns less than the preset number of columns in the preliminary image sub-blocks In the case of sub-blocks, the first number of preparatory image sub-blocks is used as the first number of image sub-blocks.

The weight sub-block obtaining module 302 obtains a weight matrix, divides the weight matrix to obtain a second number of weight vectors, and divides each of the weight vectors to obtain a first number of weight sub-blocks.

In at least one embodiment of the present application, dividing the weight matrix by the weight sub-block obtaining module 302 to obtain the second quantity of weight vectors includes: dividing the weight matrix according to the number of columns to obtain the second quantity of weight vectors. Preparing weight vector; judging whether there is a preparatory weight vector with the number of columns less than the preset number of columns in the preparatory weight vector; when there is a weight vector in the preparatory weight vector whose number of columns is less than the preset number of columns, use The preset value fills the preliminary weight vector with the number of columns less than the preset number of columns to obtain a preliminary weight vector with the number of columns equal to the preset number of columns, and the second number of preliminary weight vectors is used as the second number of all preliminary weight vectors. or when there is no weight vector whose number of columns is less than the preset number of columns in the prepared weight vector, the second number of prepared weight vectors is used as the second number of the weight vectors.

In at least one embodiment of the present application, the step of dividing each of the weight vectors by the weight sub-block obtaining module 302 to obtain the first number of weight sub-blocks includes: dividing each of the weight vectors according to the number of rows Obtain a fourth quantity of preparatory weight sub-blocks; determine whether the fourth quantity is equal to the first quantity, and when the fourth quantity is less than the first quantity, calculate the fourth quantity and the first quantity and increase the number of preparatory weight sub-blocks according to the first difference until the first number of preparatory weight sub-blocks are obtained; or when the fourth number is greater than the first number, from the fourth Selecting a first number of pre-weight sub-blocks from the number of pre-weight sub-blocks; judging whether there is a pre-weight sub-block with a number of columns less than the preset number of columns in the first number of pre-weight sub-blocks; when the first number of pre-weight sub-blocks is When there are pre-weight sub-blocks with a number of columns smaller than the preset number of columns in a number of pre-weight sub-blocks, use a preset value to fill the pre-weight sub-blocks with a number of columns smaller than the pre-set number of columns to obtain a number of columns equal to the pre-set number of columns. Set the number of columns of pre-weight sub-blocks, and use the first number of pre-weight sub-blocks as the first number of pre-weight sub-blocks; or when there is no column number smaller than the preset column in the first number of pre-weight sub-blocks When the number of preparatory weight sub-blocks is selected, the first number of preparatory weight sub-blocks is used as the first number of weight sub-blocks.

In at least one embodiment of the present application, the weight matrix is used to measure the importance of features in the image matrix.

In at least one embodiment of the present application, the weight matrix may be a matrix in an encoding layer of an autoencoder or a matrix in a convolutional layer of a convolutional neural network, and the weight matrix may be stored in In artificial intelligence chips, it is used to perform general matrix multiplication operations.

The offset quantum block obtaining module 303 obtains an offset matrix, and divides the offset matrix to obtain a second number of offset quantum blocks.

In at least one embodiment of the present application, the offset sub-block obtaining module 303 dividing the offset matrix to obtain the second number of offset sub-blocks includes: obtaining the offset matrix according to the preset The number of rows is divided to obtain a preliminary offset vector, and an offset vector is selected from the preliminary offset vector; the offset vector is divided according to the preset number of columns to obtain a fifth number of preliminary offsets. Shifting sub-blocks; judging whether the fifth quantity is equal to the second quantity, when the fifth quantity is less than the second quantity, calculate the second difference between the fifth quantity and the second quantity, and increase the number of preparatory offset quantum blocks according to the second difference until a second number of preparatory offset quantum blocks are obtained; or when the fifth number is greater than the second number, from the fifth number of Selecting a second quantity of preparatory offset quantum blocks from the preparatory offset quantum blocks; judging whether there is a second quantity of preparatory offset quantum blocks in which the number of columns is less than the preset number of columns or the number of rows is less than the preset number of rows Preparing offset quantum blocks; when there are preparatory offset quantum blocks with columns smaller than the preset number of columns or rows smaller than the preset number of rows in the second number of preliminary offset quantum blocks, use a preset value to fill The number of columns is less than the preset number of columns or the number of rows is less than the number of pre-set offset sub-blocks to obtain the number of columns is equal to the number of pre-set columns and the number of rows is equal to the number of pre-set offset sub-blocks, the second The number of preliminary offset quantum blocks is used as the second number of offset quantum blocks; or when there is no column number less than the preset number of columns or the number of rows less than the preset number of columns in the second number of preliminary offset quantum blocks When the number of rows of preliminary offset quantum blocks is used, the second number of the preliminary offset quantum blocks is used as the second number of offset quantum blocks.

In at least one embodiment of the present application, the offset matrix is used to measure the difficulty of generating positive excitation or negative excitation by the neural network.

In at least one embodiment of the present application, the offset matrix may be a matrix in an encoding layer of an autoencoder, or may be a matrix in a convolutional layer of a convolutional neural network, and the offset Matrices can be stored in AI chips for general-purpose matrix multiplication.

The image feature calculation module 304 performs vector multiplication on the first number of image sub-blocks and the first number of weight sub-blocks of each of the weight vectors to obtain a second number of feature sub-blocks, and the second number of The feature sub-blocks are added with the second number of offset sub-blocks to obtain image feature sub-blocks, and the image feature sub-blocks constitute image features.

In at least one embodiment of the present application, in at least one embodiment of the present application, each of the image sub-block, the weight sub-block, and the offset sub-block has a preset number of rows equal to The preset number of columns, the preset number of columns is the same as the preset number of rows.

In this embodiment, by dividing the image matrix, the weight matrix and the deviation matrix into sub-blocks for operation, the calculation amount of the convolution operation in the neural network in the process of image feature extraction is reduced, thereby improving the accuracy of the image in the image. The efficiency of feature extraction.

Example 3

FIG. 3 is a schematic diagram of an electronic device 6 in an embodiment of the present application.

The electronic device 6 includes a memory 61 , a processor 62 and a computer program 63 stored in the memory 61 and executable on the processor 62 . When the processor 62 executes the computer program 63 , the steps in the above-mentioned embodiment of the image feature extraction method are implemented, such as steps S11 to S14 shown in FIG. 1 . Alternatively, when the processor 62 executes the computer program 63 , the functions of each module/unit in the above-mentioned embodiment of the image feature extraction apparatus, such as modules 301 to 304 in FIG. 2 , are realized.

Exemplarily, the computer program 63 can be divided into one or more modules/units, and the one or more modules/units are stored in the memory 61 and executed by the processor 62 , to complete this application. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the computer program 63 in the electronic device 6 execution process. For example, the computer program 63 can be divided into an image sub-block acquisition module 301, a weight sub-block acquisition module 302, an offset sub-block acquisition module 303 and an image feature calculation module 304 in FIG. For the specific functions of the module, see Embodiment 2.

In this embodiment, the electronic device 6 may be a computing device such as a desktop computer, a notebook, a palmtop computer, a server, and a cloud terminal device. Those skilled in the art can understand that the schematic diagram is only an example of the electronic device 6, and does not constitute a limitation to the electronic device 6, and may include more or less components than the one shown, or combine some components, or different Components such as the electronic device 6 may also include input and output devices, network access devices, bus bars, and the like.

The processor 62 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs) ), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor 62 can also be any conventional processor, etc. The processor 62 is the control center of the electronic device 6, and uses various interfaces and lines to connect the entire electronic device 6. various parts.

The memory 61 can be used to store the computer programs 63 and/or modules/units, and the processor 62 runs or executes the computer programs and/or modules/units stored in the memory 61, And call the data stored in the memory 61 to realize various functions of the electronic device 6 . The memory 61 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; storage data The area may store data (such as audio data, phone book, etc.) created according to the use of the electronic device 6, and the like. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), Secure Digital (SD) card, flash memory card (Flash Card), at least one disk memory device, flash memory device, or other volatile solid state memory device.

If the modules/units integrated in the electronic device 6 are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the methods of the above embodiments, and can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, When the computer program is executed by the processor, the steps of the above method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of original code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-Only Memory); Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and other division methods may be used in actual implementation.

In addition, each functional module in each embodiment of the present application may be integrated in the same processing module, or each module may exist physically alone, or two or more modules may be integrated in the same module. The above-mentioned integrated modules can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, but that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be regarded as exemplary in all respects, And, without limitation, the scope of the invention is defined by the appended claims rather than the foregoing description, and therefore all changes that come within the meaning and range of equivalents to the claimed scope are intended to be embraced therein. Any reference signs in the patentable scope should not be construed as limiting the claimed scope. Furthermore, it is clear that the word "comprising" does not exclude other modules or steps, and the singular does not exclude the plural. Multiple modules or electronic devices stated in the scope of the electronic device patent application can also be implemented by the same module or electronic device through software or hardware. The terms first, second, etc. are used to denote names and do not denote any particular order.

To sum up, the present invention complies with the requirements of an invention patent, and a patent application can be filed in accordance with the law. However, the above descriptions are only the preferred embodiments of the present invention, and for those who are familiar with the techniques of this case, equivalent modifications or changes made in accordance with the creative spirit of this case shall be included in the scope of the following patent application.

S11~S14: Steps

Claims (8)

An image feature extraction method, applied to electronic equipment, wherein the method comprises: acquiring an image, converting the image into an image matrix, dividing the image matrix to obtain an image vector, dividing the image vector to obtain a first number of image sub-blocks; obtain a weight matrix, divide the weight matrix to obtain a second number of weight vectors, divide each of the weight vectors to obtain a first number of weight sub-blocks, wherein the weight matrix is a matrix in the coding layer of the self-encoder, or a matrix in the convolutional layer of the convolutional neural network, the weight matrix is used to measure the importance of the features in the image matrix; obtain the offset matrix, divide Obtaining a second number of offset sub-blocks from the offset matrix includes: dividing the weight matrix according to the number of columns to obtain a second number of preparatory weight vectors; judging whether there is a number of columns in the preparatory weight vector A prepared weight vector less than the preset number of columns; when there is a weight vector with a number of columns less than the preset number of columns in the prepared weight vector, use the preset value to fill the number of columns less than the preset number of columns. The preparatory weight vector of the number of columns obtains a preparatory weight vector with the number of columns equal to the preset number of columns, and the preparatory weight vector of the second quantity is used as the weight vector of the second quantity; or when there is no column in the preparatory weight vector When the number of weight vectors is less than the preset number of columns, the second number of preparatory weight vectors is used as the second number of weight vectors, wherein the offset matrix is a matrix in the coding layer of the self-encoder , or a matrix in the convolutional layer of the convolutional neural network, the offset matrix is used to measure the difficulty of the convolutional neural network or the autoencoder to generate positive or negative excitation; for the first number of graphs The image sub-blocks perform vector multiplication with the first number of weight sub-blocks of each of the weight vectors to obtain a second number of feature sub-blocks, and the second number of feature sub-blocks is combined with the second number of offset sub-blocks. The blocks are added to obtain image feature sub-blocks, and the image feature sub-blocks form image features, wherein, Each of the image sub-block, the weight sub-block, and the offset sub-block has a preset number of rows and a preset number of columns, and the preset number of columns is the same as the preset number of rows. The image feature extraction method according to claim 1, wherein the dividing the image matrix to obtain the image vector comprises: dividing the image matrix according to the preset number of rows to obtain a set of preliminary image vectors , the preliminary image vector set includes at least one preliminary image vector; determine whether there is a preliminary image vector with a row number less than the preset number of rows in the preliminary image vector set; when there is a preliminary image vector in the preliminary image vector When the image vector with the number of lines is less than the preset number of lines, use the preset value to fill the preliminary image vector with the number of lines less than the preset number of lines to obtain a preliminary image vector with the number of lines equal to the preset number of lines an image vector, using the preliminary image vector as the image vector; or when there is no image vector with a row number less than the preset number of rows in the preliminary image vector, the preliminary image vector vector as the image vector. The image feature extraction method according to claim 1, wherein dividing the image vector to obtain the first number of image sub-blocks comprises: dividing the image vector according to the preset number of columns to obtain the first number of sub-blocks. number of preliminary image sub-blocks; determine whether there are preliminary image sub-blocks with a number of columns less than the preset number of columns in the preliminary image sub-blocks; when the number of columns in the preliminary image sub-block is less than the When there are pre-set image sub-blocks with a preset number of columns, use a preset value to fill the pre-image sub-blocks with a number of columns less than the preset number of columns to obtain pre-image sub-blocks with a number of columns equal to the preset number of columns, A number of preparatory image sub-blocks are used as the first number of image sub-blocks; or when there is no preparatory image sub-block with a column number smaller than the preset number of columns in the preparatory image sub-block, use The first number of preliminary image sub-blocks are used as the first number of image sub-blocks. The image feature extraction method according to claim 1, wherein dividing each of the weight vectors to obtain a first number of weight sub-blocks comprises: dividing each of the weight vectors according to the number of rows to obtain a fourth number of sub-blocks The preparatory weight sub-block of the value, and increase the number of preparatory weight sub-blocks according to the first difference value until the first number of preparatory weight sub-blocks are obtained; or when the fourth number is greater than the first number, from the fourth number of preparatory weights Selecting a first number of pre-weight sub-blocks in the sub-blocks; judging whether there are pre-weight sub-blocks with a number of columns less than the preset number of columns in the first number of pre-weight sub-blocks; when the first number of pre-weight sub-blocks exist When there are pre-weight sub-blocks with a number of columns less than the preset number of columns in the weight sub-block, use a preset value to fill the pre-weight sub-block with a number of columns less than the preset number of columns to obtain a pre-weight sub-block with a number of columns equal to the preset number of columns. Preparing weight sub-blocks, using the first number of pre-weight sub-blocks as the first number of weight sub-blocks; or when there is no pre-weight with a column number smaller than the preset number of columns in the first number of pre-weight sub-blocks In the case of sub-blocks, the first number of preparatory weight sub-blocks is used as the first number of weight sub-blocks. The image feature extraction method according to claim 1, wherein the dividing the offset matrix to obtain the second number of offset sub-blocks comprises: performing the offset matrix according to the preset number of rows. Divide to obtain a preliminary offset vector, and select an offset vector from the preliminary offset vector; divide the offset vector according to the preset number of columns to obtain a fifth number of preliminary offset quantum blocks ; Determine whether the fifth quantity is equal to the second quantity, when the fifth quantity is less than the second quantity, calculate the second difference between the fifth quantity and the second quantity, and calculate the second difference according to the The second difference increases the number of preparatory offset quantum blocks until a second number of preparatory offset quantum blocks is obtained; or when the fifth number is greater than the second number, from the fifth number of preparatory offset quantum blocks Select a second number of preliminary offset sub-blocks in the block; determine whether there is a preliminary offset sub-block with a number of columns smaller than the preset number of columns or a number of rows smaller than the preset number of rows in the second number of preliminary offset sub-blocks block; when there are preliminary offset sub-blocks with the number of columns less than the preset number of columns or the number of rows less than the preset number of rows in the second number of preliminary offset sub-blocks, use the preset value to fill in the number of columns less than the preset number of rows. The pre-offset quantum blocks with the preset number of columns or rows less than the preset number of rows obtain the pre-offset sub-blocks with the number of columns equal to the preset number of columns and the number of rows equal to the preset number of rows, and the second number of pre-offset quantum blocks are Shift sub-blocks as the second number of offset sub-blocks; or when there is no preparation with a column number smaller than the preset number of columns or a row number smaller than the preset number of rows in the second number of preliminary offset sub-blocks When offsetting the quantum blocks, the second quantity of the preparatory offset quantum blocks is used as the second quantity of offset quantum blocks. An image feature extraction device, wherein the device comprises: an image sub-block acquisition module for acquiring an image, converting the image into an image matrix, dividing the image matrix to obtain an image vector, Dividing the image vector to obtain a first number of image sub-blocks; a weight sub-block obtaining module is used to obtain a weight matrix, dividing the weight matrix to obtain a second number of weight vectors, and dividing each of the weight vectors to obtain a first number of weight sub-blocks, wherein the weight matrix is a matrix in an encoding layer of an autoencoder, or a matrix in a convolutional layer of a convolutional neural network, and the weight matrix is used to measure the image The importance of the features in the matrix; the offset quantum block obtaining module is used to obtain the offset matrix, and dividing the offset matrix to obtain a second number of offset quantum blocks, including: obtaining the weight matrix according to the Dividing the number of columns to obtain a second number of preliminary weight vectors; judging whether there is a number of columns in the preliminary weight vector that is smaller than the preset number of columns When there is a weight vector with a number of columns less than the preset number of columns in the preliminary weight vector, use the preset value to fill the preliminary weight vector with the number of columns less than the preset number of columns. Obtaining a preparatory weight vector with a number of columns equal to the preset number of columns, and using the second number of preparatory weight vectors as the second number of the weight vectors; or when there is no column number in the preparatory weight vector that is smaller than the preset number of columns When the number of columns is the weight vector, the second number of preparatory weight vectors is used as the second number of the weight vectors, wherein the offset matrix is a matrix in the coding layer of the self-encoder, or a convolutional neural network. A matrix in the convolutional layer of the network, the offset matrix is used to measure the difficulty of the convolutional neural network or the self-encoder to generate positive excitation or negative excitation; the image feature calculation module is used for the first The number of image sub-blocks and the first number of weight sub-blocks of each of the weight vectors are vector multiplied to obtain a second number of feature sub-blocks, and the second number of feature sub-blocks is combined with the second number of feature sub-blocks. The offset quantum blocks are added to obtain image feature sub-blocks, and the image feature sub-blocks constitute image features. An electronic device, wherein the electronic device comprises: a memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to realize the image according to any one of claim 1 to 5 Feature extraction method. A storage medium having a computer program stored thereon, wherein: the computer program implements the image feature extraction method according to any one of claim 1 to 5 when the computer program is executed by a processor.

Images