CN109948700A

CN109948700A - Method and apparatus for generating characteristic pattern

Info

Publication number: CN109948700A
Application number: CN201910209390.6A
Authority: CN
Inventors: 喻冬东; 王长虎
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Douyin Vision Co Ltd; Douyin Vision Beijing Co Ltd
Priority date: 2019-03-19
Filing date: 2019-03-19
Publication date: 2019-06-28
Anticipated expiration: 2039-03-19
Also published as: CN109948700B

Abstract

Embodiment of the disclosure discloses the method and apparatus for generating characteristic pattern.One specific embodiment of this method includes: acquisition target image, and determines the characteristic pattern of target image；The modulation of single order spatial attention, characteristic pattern after being modulated are carried out to characteristic pattern；First process of convolution is carried out to characteristic pattern after modulation, obtains eigenmatrix after preset number the first convolution, wherein the preset number channel one-to-one correspondence that the characteristic pattern obtained after eigenmatrix and the first process of convolution after the first convolution includes；The pixel in pixel for including for target image, from eigenmatrix, determining the corresponding pixel characteristic vector of the pixel after preset number the first convolution；Based on obtained pixel characteristic vector, pixel relationship matrix is determined；Based on pixel relationship matrix, characteristic pattern after modulation is converted, generates characteristic pattern after transformation.The embodiment can allow characteristic pattern after obtained transformation more fully to characterize the feature of target image, help to improve the accuracy identified to image, and improve the accuracy from extracting target from images object images.

Description

Method and apparatus for generating characteristic pattern

Technical field

Embodiment of the disclosure is related to field of computer technology, and in particular to the method and apparatus for generating characteristic pattern.

Background technique

Existing convolutional neural networks, when extracting the feature of image, usually according to the size of convolution kernel, in image A part of region pixel for including analyzed, such as by the corresponding characteristic of each pixel multiplied by corresponding power Weight, to obtain new characteristic.The characteristic of each pixel is individually determining.

Summary of the invention

Embodiment of the disclosure proposes the method and apparatus for generating characteristic pattern, and the method for image for identification And device.

In a first aspect, embodiment of the disclosure provides a kind of method for generating characteristic pattern, this method comprises: obtaining Target image, and determine the characteristic pattern of target image, wherein characteristic pattern includes at least one channel, and each channel corresponds to One eigenmatrix；The modulation of single order spatial attention, characteristic pattern after being modulated are carried out to characteristic pattern；To characteristic pattern after modulation into The first process of convolution of row obtains eigenmatrix after the first convolution of preset number, wherein eigenmatrix and first after the first convolution The preset number channel that the characteristic pattern obtained after process of convolution includes corresponds；In the pixel for including for target image Pixel, from eigenmatrix, determining the corresponding pixel characteristic vector of the pixel after preset number the first convolution；It is based on Obtained pixel characteristic vector, determines pixel relationship matrix, wherein the element that pixel relationship matrix includes is for characterizing target The incidence relation between pixel that image includes；Based on pixel relationship matrix, characteristic pattern after modulation is converted, generates and becomes Change rear characteristic pattern.

In some embodiments, it is based on obtained pixel characteristic vector, determines pixel relationship matrix, comprising: by gained The pixel characteristic vector combination arrived, obtains matrix after the first combination；Second process of convolution is carried out to characteristic pattern after modulation, is obtained pre- If eigenmatrix after number the second convolution, wherein the feature obtained after eigenmatrix and the second process of convolution after the second convolution The preset number channel that figure includes corresponds；For the in eigenmatrix after obtained preset number the second convolution Eigenmatrix after second convolution is converted to first passage feature vector by eigenmatrix after two convolution；By obtained first The combination of channel characteristics vector, obtains matrix after the second combination；Matrix multiple after combining matrix after the first combination with second, is based on The matrix obtained after multiplication generates pixel relationship matrix.

In some embodiments, pixel relationship matrix is generated based on the matrix that obtains after multiplication, comprising: to obtaining after multiplication The matrix element that includes be normalized, obtain pixel relationship matrix.

In some embodiments, characteristic pattern includes preset number channel after modulation, and each channel corresponds to a feature Matrix；And it is based on pixel relationship matrix, characteristic pattern after modulation is converted, characteristic pattern after transformation is generated, comprising: for adjusting The corresponding eigenmatrix in the channel is converted to second channel spy by the channel in preset number channel that characteristic pattern includes after system Levy vector；By obtained second channel combination of eigenvectors, matrix after third combination is obtained；Matrix and picture after combining third Plain relational matrix is multiplied, and characteristic pattern after converting is generated based on obtained matrix after being multiplied.

Second aspect, embodiment of the disclosure provide a kind of method of image for identification, this method comprises: obtain to Identify image, wherein images to be recognized includes target object image；By images to be recognized input convolutional Neural net trained in advance Network is exported for characterizing the location information of position of the target object image in images to be recognized in images to be recognized and being used for Characterize the classification information of classification belonging to target object image, wherein convolutional neural networks include convolutional layer and classification layer, convolution Method of the layer for executing any embodiment description in above-mentioned first aspect using images to be recognized, generates characteristic pattern after transformation, Layer of classifying is used to classify to the pixel that images to be recognized includes based on characteristic pattern after transformation, be generated classification information and position Confidence breath.

In some embodiments, this method further include: be based on location information, target object figure is extracted from images to be recognized Picture and display.

The third aspect, embodiment of the disclosure provide a kind of for generating the device of characteristic pattern, which includes: first Acquiring unit is configured to obtain target image, and determines the characteristic pattern of target image, wherein characteristic pattern includes at least one A channel, each channel correspond to an eigenmatrix；Modulation unit is configured to carry out single order spatial attention to characteristic pattern Modulation, characteristic pattern after being modulated；Convolution unit is configured to carry out the first process of convolution to characteristic pattern after modulation, obtain pre- If eigenmatrix after number the first convolution, wherein the feature obtained after eigenmatrix and the first process of convolution after the first convolution The preset number channel that figure includes corresponds；First determination unit is configured to the pixel for including for target image In pixel, from eigenmatrix, determining the corresponding pixel characteristic vector of the pixel after preset number the first convolution；The Two determination units are configured to determine pixel relationship matrix, wherein pixel relationship square based on obtained pixel characteristic vector The element that battle array includes is used to characterize the incidence relation between the pixel that target image includes；Converter unit is configured to be based on Pixel relationship matrix converts characteristic pattern after modulation, generates characteristic pattern after transformation.

In some embodiments, the second determination unit includes: the first composite module, is configured to obtained pixel is special Vector combination is levied, matrix after the first combination is obtained；Convolution module is configured to carry out at the second convolution characteristic pattern after modulation Reason obtains eigenmatrix after the second convolution of preset number, wherein obtains after eigenmatrix and the second process of convolution after the second convolution To characteristic pattern include preset number channel correspond；First conversion module is configured to for obtained default Eigenmatrix after second convolution is converted to the by eigenmatrix after the second convolution after number the second convolution in eigenmatrix One channel characteristics vector；Second composite module is configured to obtained first passage combination of eigenvectors obtaining second group Matrix after conjunction；First generation module, matrix multiple after being configured to combine matrix after the first combination with second, after being multiplied Obtained matrix generates pixel relationship matrix.

In some embodiments, generation module is further configured to: the element for including to the matrix obtained after multiplication into Row normalized obtains pixel relationship matrix.

In some embodiments, characteristic pattern includes preset number channel after modulation, and each channel corresponds to a feature Matrix；And converter unit includes: the second conversion module, is configured to lead to the preset number that characteristic pattern includes after modulation The corresponding eigenmatrix in the channel is converted to second channel feature vector by the channel in road；Third composite module, is configured to By obtained second channel combination of eigenvectors, matrix after third combination is obtained；Second generation module is configured to third Matrix and pixel relationship matrix multiple after combination generate characteristic pattern after converting based on obtained matrix after being multiplied.

Fourth aspect, embodiment of the disclosure provide a kind of device of image for identification, which includes: second to obtain Unit is taken, is configured to obtain images to be recognized, wherein images to be recognized includes target object image；Output unit is configured At the convolutional neural networks that images to be recognized input is trained in advance, export for characterizing the target object figure in images to be recognized Classification information as the location information of the position in images to be recognized and for characterizing classification belonging to target object image, In, convolutional neural networks include convolutional layer and classification layer, and convolutional layer is used to execute in above-mentioned first aspect using images to be recognized The method of any embodiment description, generates characteristic pattern after transformation, and classification layer is used for based on characteristic pattern after transformation, to images to be recognized Including pixel classify, generate classification information and location information.

In some embodiments, device further include: display unit is configured to based on location information, from figure to be identified Target object image and display are extracted as in.

5th aspect, embodiment of the disclosure provide a kind of electronic equipment, which includes: one or more places Manage device；Storage device is stored thereon with one or more programs；When one or more programs are held by one or more processors Row, so that one or more processors realize the method as described in implementation any in first aspect or second aspect.

6th aspect, embodiment of the disclosure provide a kind of computer-readable medium, are stored thereon with computer program, The method as described in implementation any in first aspect or second aspect is realized when the computer program is executed by processor.

The method and apparatus for generating characteristic pattern that embodiment of the disclosure provides, by acquisition target image, and It determines the characteristic pattern of target image, then the modulation of single order spatial attention is carried out to characteristic pattern, characteristic pattern after being modulated is then right Characteristic pattern carries out the first process of convolution after modulation, obtains eigenmatrix after the first convolution of preset number, wherein after the first convolution Preset number channel that the characteristic pattern obtained after eigenmatrix and the first process of convolution includes corresponds, then from obtained After first convolution in eigenmatrix, the corresponding pixel characteristic vector of pixel that target image includes is determined, then be based on picture Plain feature vector determines pixel relationship matrix, is finally based on pixel relationship matrix, converts to characteristic pattern after modulation, generates Characteristic pattern after transformation.Since characteristic pattern is generated according to identified pixel relationship matrix after transformation, feature after transformation Figure can be used for characterizing the relationship between the pixel that target image includes, so that characteristic pattern can be more after obtained transformation The feature for comprehensively characterizing target image helps to improve the accuracy identified to image, and improves and mention from image Take the accuracy of target object image.

Detailed description of the invention

By reading a detailed description of non-restrictive embodiments in the light of the attached drawings below, the disclosure is other Feature, objects and advantages will become more apparent upon:

Fig. 1 is that one embodiment of the disclosure can be applied to exemplary system architecture figure therein；

Fig. 2 is according to an embodiment of the present disclosure for generating the flow chart of one embodiment of the method for characteristic pattern；

Fig. 3 is according to an embodiment of the present disclosure for generating the schematic diagram of an application scenarios of the method for characteristic pattern；

Fig. 4 is the flow chart of one embodiment of the method for image for identification according to an embodiment of the present disclosure；

Fig. 5 is according to an embodiment of the present disclosure for generating the structural representation of one embodiment of the device of characteristic pattern Figure；

Fig. 6 is the structural schematic diagram of one embodiment of the device of image for identification according to an embodiment of the present disclosure；

Fig. 7 is adapted for the structural schematic diagram for realizing the terminal device of embodiment of the disclosure.

Specific embodiment

The disclosure is described in further detail with reference to the accompanying drawings and examples.It is understood that this place is retouched The specific embodiment stated is used only for explaining that correlation is open, rather than the restriction to the disclosure.It also should be noted that in order to Convenient for description, is illustrated only in attached drawing and disclose relevant part to related.

It should be noted that in the absence of conflict, the feature in embodiment and embodiment in the disclosure can phase Mutually combination.The disclosure is described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

Fig. 1 is shown can be using embodiment of the disclosure for generating the method for characteristic pattern or for generating characteristic pattern Device, and the method for image or for identification exemplary system architecture 100 of the device of image for identification.

As shown in Figure 1, system architecture 100 may include terminal device 101,102,103, network 104 and server 105. Network 104 between terminal device 101,102,103 and server 105 to provide the medium of communication link.Network 104 can be with Including various connection types, such as wired, wireless communication link or fiber optic cables etc..

User can be used terminal device 101,102,103 and be interacted by network 104 with server 105, to receive or send out Send message etc..Various telecommunication customer end applications can be installed on terminal device 101,102,103, such as image processing application, Video playing application, searching class application, instant messaging tools, social platform software etc..

Terminal device 101,102,103 can be hardware, be also possible to software.When terminal device 101,102,103 is hard When part, it can be various electronic equipments.When terminal device 101,102,103 is software, above-mentioned electronic equipment may be mounted at In.Multiple softwares or software module (such as providing the software of Distributed Services or software module) may be implemented into it, Single software or software module may be implemented into.It is not specifically limited herein.

Server 105 can be to provide the server of various services, such as to the figure that terminal device 101,102,103 uploads As the backstage image processing server handled.Image processing server available image in backstage is handled, and is obtained Processing result (such as characteristic pattern of image).

It should be noted that can be by server for generating the method for characteristic pattern provided by embodiment of the disclosure 105 execute, can also be executed by terminal device 101,102,103, correspondingly, the device for generating characteristic pattern can be set in In server 105, also it can be set in terminal device 101,102,103.In addition, being used for provided by embodiment of the disclosure The method of identification image can be executed by server 105, can also be executed by terminal device 101,102,103, correspondingly, be used for The device of identification image can be set in server 105, also can be set in terminal device 101,102,103.

It should be noted that server can be hardware, it is also possible to software.When server is hardware, may be implemented At the distributed server cluster that multiple servers form, individual server also may be implemented into.It, can when server is software To be implemented as multiple softwares or software module (such as providing the software of Distributed Services or software module), also may be implemented At single software or software module.It is not specifically limited herein.

It should be understood that the number of terminal device, network and server in Fig. 1 is only schematical.According to realization need It wants, can have any number of terminal device, network and server.It is not required in the image handled from the feelings remotely obtained Under condition, above system framework can not include network, only include server or terminal device.

With continued reference to Fig. 2, the process of one embodiment of the method for generating characteristic pattern according to the disclosure is shown 200.The method for being used to generate characteristic pattern, comprising the following steps:

Step 201, target image is obtained, and determines the characteristic pattern of target image.

In the present embodiment, for generating executing subject (such as server shown in FIG. 1 or the terminal of the method for characteristic pattern Equipment) target image can be obtained from long-range, or from local by wired connection mode or radio connection.Wherein, mesh Logo image is to handle it, to generate the image of its corresponding characteristic pattern.For example, target image can be above-mentioned execution The image that the image for the camera shooting that main body includes or above-mentioned executing subject are extracted from preset image collection.

Above-mentioned executing subject may further determine that the characteristic pattern of target image.Wherein, characteristic pattern (feature map) is used In the feature (such as color characteristic, gray feature etc.) of characterization image.In general, characteristic pattern includes at least one channel, Mei Getong Road is used to characterize a kind of feature of image, meanwhile, each channel corresponds to an eigenmatrix, each member in eigenmatrix Element, a pixel for including corresponding to target image.

Above-mentioned executing subject can determine the characteristic pattern of target image in various manners.As an example, above-mentioned execution master Each pixel that body can include according to target image color value (including R (Red, red) value, G (Green, green) value, B (Blue, blue) value), generate the characteristic pattern including three channels (the respectively channel R, the channel G, channel B), each channel pair Ying Yuyi eigenmatrix, element therein are the color value of corresponding color.

For another example target image can be inputted preset convolutional neural networks, convolutional neural networks by above-mentioned executing subject Including convolutional layer can extract the feature of target image, generate characteristic pattern.In general, convolutional layer may include at least one convolution Core, each convolution kernel can be used for generating an eigenmatrix.It should be noted that in general, convolutional neural networks may include Multiple convolutional layers, characteristic pattern used in the present embodiment can be the characteristic pattern that any convolutional layer generates.

Step 202, the modulation of single order spatial attention, characteristic pattern after being modulated are carried out to characteristic pattern.

In the present embodiment, above-mentioned executing subject can carry out the modulation of single order spatial attention to characteristic pattern, be modulated Characteristic pattern afterwards.Wherein, single order spatial attention modulation (Spatial Attention) refers to, by characteristic pattern corresponding at least one A eigenmatrix is remapped to multiple vectors, and each vector therein corresponds to the pixel that target image includes, then Using preset function, operation (such as the operations such as weighting, classification, convolution) are carried out to these vectors, finally turn operation result New characteristic pattern is changed to as characteristic pattern after modulation, wherein characteristic pattern includes at least one channel after modulation, and each channel is corresponding In an eigenmatrix.Since above-mentioned operation is to carry out linear operation to above-mentioned multiple vectors, referred to as single order space is infused Power of anticipating modulation.

The characteristic pattern after obtained modulation after the modulation of single order spatial attention, can be used for characterizing target image includes Each pixel various features.In practice, characteristic pattern carries out the pixel that image includes after modulation usually can be used The operation such as classification, so that the modulation of single order spatial attention can be applied to the fields such as image recognition, image classification.

Step 203, the first process of convolution is carried out to characteristic pattern after modulation, obtains feature square after the first convolution of preset number Battle array.

In the present embodiment, above-mentioned executing subject can carry out the first process of convolution to characteristic pattern after modulation, be preset Eigenmatrix after the first convolution of number.Wherein, the characteristic pattern obtained after eigenmatrix and the first process of convolution after the first convolution Including preset number channel correspond.In general, above-mentioned executing subject can use preset preset number convolution kernel At least one eigenmatrix corresponding to characteristic pattern after modulation carries out the first process of convolution, to obtain the preset number first volume Eigenmatrix after product.Convolution kernel is usually the form of matrix, and element therein is preset weighted value, can be with using weighted value At least one eigenmatrix corresponding to characteristic pattern after modulation carries out convolution algorithm.It should be noted that the power that convolution kernel includes Weight values can be pre-set, be also possible to advance with machine learning method, to convolutional neural networks belonging to convolution kernel Determined by after being trained.In the present embodiment, above-mentioned preset number, which is typically larger than, is equal to 2.

Step 204, the pixel in pixel for including for target image, from feature after the first convolution of preset number In matrix, the corresponding pixel characteristic vector of the pixel is determined.

In the present embodiment, the pixel in pixel for including for target image, above-mentioned executing subject can be from pre- If determining the corresponding pixel characteristic vector of the pixel in eigenmatrix after the first convolution of number.

In general, the element after each first convolution in eigenmatrix, the pixel for including with target image is corresponded.It is right In a pixel, above-mentioned executing subject can be from after each first convolution in eigenmatrix, will member corresponding with the pixel Element extracts and group is combined into a vector as pixel characteristic vector.As an example it is supposed that there is feature square after C the first convolution Gust, eigenmatrix is the matrix of H row W column after each first convolution.Wherein, H is the line number for the pixel that target image includes, W For the columns for the pixel that target image includes.The then pixel characteristic vector of available N number of C dimension, wherein N=H × W.

Step 205, it is based on obtained pixel characteristic vector, determines pixel relationship matrix.

In the present embodiment, above-mentioned executing subject can be based on obtained pixel characteristic vector, determine pixel relationship square Battle array.Wherein, the element that pixel relationship matrix includes is used to characterize relationship (such as the pixel between the pixel that target image includes Color, gray scale, brightness etc. between relationship).

Specifically, as an example, above-mentioned executing subject can combine pixel characteristic vector, feature square after being combined Battle array, then by eigenmatrix after combination with combine after eigenmatrix transposed matrix multiplication, the matrix obtained after multiplication is determined as Pixel relationship matrix.Continue the example in above-mentioned steps 204, the pixel characteristic vector of N number of C dimension can be combined into the group of N row C column Eigenmatrix after conjunction, the transposed matrix of eigenmatrix is C row N column after combination, then the matrix obtained after being multiplied is that N row N is arranged Pixel relationship matrix.Every row of eigenmatrix corresponds to a pixel after said combination, and each column of above-mentioned transposed matrix is corresponding In a pixel, therefore, each element in pixel relationship matrix in this example can correspond respectively to two pixels, So as to for characterizing the incidence relation between pixel.Such as the corresponding pixel characteristic vector of two pixels is more similar (color value of i.e. two pixels is more similar), then in pixel relationship matrix, corresponding with the two pixels element is closer to it In the pixel characteristic vector element that includes quadratic sum.

In some optional implementations of the present embodiment, above-mentioned executing subject can be based on obtained pixel characteristic Vector determines pixel relationship matrix in accordance with the following steps:

Step 1 combines obtained pixel characteristic vector, obtains matrix after the first combination.

As an example, the pixel characteristic vector of N number of C dimension can be combined into matrix after the first combination that N row C is arranged.

Step 2 carries out the second process of convolution to characteristic pattern after modulation, obtains feature square after the second convolution of preset number Battle array.

Specifically, it is corresponding to characteristic pattern after modulation to can use preset preset number convolution kernel for above-mentioned executing subject At least one eigenmatrix carries out the second process of convolution, to obtain eigenmatrix after the second convolution of preset number, wherein the The preset number channel that the characteristic pattern obtained after eigenmatrix and the second process of convolution after two convolution includes corresponds.It needs Illustrate, convolution kernel used herein above can be different from convolution kernel used in above-mentioned steps 203.Therefore, here The feature that eigenmatrix characterizes after second convolution is different from the feature of eigenmatrix characterization after the first convolution in step 203.

Step 3, for feature square after the second convolution in eigenmatrix after the second convolution of obtained preset number Battle array, is converted to first passage feature vector for eigenmatrix after second convolution.Wherein, first passage feature vector includes second Whole elements of eigenmatrix after convolution.As an example it is supposed that eigenmatrix is H row W column after some second convolution, then can incite somebody to action Its vector for being converted to N-dimensional is as first passage feature vector, wherein N=H × W.

Obtained first passage combination of eigenvectors is obtained matrix after the second combination by step 4.

Continue the example in above-mentioned steps three, it is assumed that preset number C, then the first passage feature of available C N-dimensional Vector, after the first passage combination of eigenvectors of C N-dimensional, matrix after the second combination of available C row N column.

Step 5, matrix multiple after combining matrix after the first combination with second, is generated based on the matrix obtained after multiplication Pixel relationship matrix.Here, the method due to using matrix multiple, the operation to pixel characteristic vector is no longer line Property, the modulation of second order spatial attention can be referred to as here.

Continue the example in above-mentioned steps four, after the first combination of N row C column matrix with C row N is arranged second combine after matrix After multiplication, which can be determined as pixel relationship matrix by the matrix of available N row N column.Obtained by this implementation Pixel relationship matrix, generated by being then based on eigenmatrix after the second convolution, therefore, can be used for characterizing between pixel The relationship of different characteristic.

In some optional implementations of the present embodiment, above-mentioned executing subject can be to the matrix packet obtained after multiplication The element included is normalized, and obtains pixel relationship matrix.Obtained pixel relationship matrix after normalized, packet The element included is between 0 to 1, therefore, can be as the weight for extracting other features, to help to make to mention Other features taken can reflect the relationship between pixel.The algorithm of above-mentioned normalized can include but is not limited to following It is a kind of: z-score standardized algorithm, softmax algorithm.

Step 206, it is based on pixel relationship matrix, characteristic pattern after modulation is converted, generates characteristic pattern after transformation.

In the present embodiment, above-mentioned executing subject can be based on pixel relationship matrix, convert to characteristic pattern after modulation, Generate characteristic pattern after converting.Wherein, characteristic pattern can be used for characterizing the pass between each pixel that target image includes after transformation System.

In some optional implementations of the present embodiment, characteristic pattern includes preset number channel after modulation, each Channel corresponds to an eigenmatrix.This step can execute as follows:

The channel in preset number channel for including firstly, for characteristic pattern after modulation, by the corresponding feature in the channel Matrix conversion is second channel feature vector.As an example it is supposed that some eigenmatrix is H row W column, then can be converted into The vector of N-dimensional is as second channel feature vector, wherein N=H × W.

Then, by obtained second channel combination of eigenvectors, matrix after third combination is obtained.As an example it is supposed that Preset number is C, then the second channel feature vector of available C N-dimensional, by the second channel combination of eigenvectors of C N-dimensional Afterwards, matrix after the third combination of available C row N column.

Finally, matrix and pixel relationship matrix multiple after combining third, are generated based on obtained matrix after being multiplied and are become Change rear characteristic pattern.It continues the example presented above, the pixel relationship matrix multiple of matrix and N row N column after the third combination of C row N column can be with Obtain the matrix of C row N column.For every row in obtained matrix after being multiplied, which corresponds to a channel, can be by the row packet The N column element included is converted to the eigenmatrix of H row W column again.So as to obtain corresponding to preset number eigenmatrix Characteristic pattern after transformation.

Optionally, it when the number and above-mentioned preset number difference in the channel that characteristic pattern includes after modulation, can use pre- If preset number convolution kernel (be different from above-mentioned for generating after the first convolution eigenmatrix after eigenmatrix and the second convolution Convolution kernel used), process of convolution is carried out to characteristic pattern after modulation, obtains preset number eigenmatrix.Using obtained Preset number eigenmatrix generates characteristic pattern after transformation according to above-mentioned optional implementation.

It is that one of the application scenarios of the method according to the present embodiment for generating characteristic pattern shows with continued reference to Fig. 3, Fig. 3 It is intended to.In the application scenarios of Fig. 3, electronic equipment 301 obtains pre-stored target image 302 from local first.It recycles Preset convolutional neural networks carry out feature extraction to target image 302, obtain the characteristic pattern of target image 302.Wherein, it wraps Include at least one channel, each channel corresponds to an eigenmatrix (as shown in 303 in figure).Then, electronic equipment 301 is right Characteristic pattern carries out the modulation of single order spatial attention, characteristic pattern after being modulated, wherein characteristic pattern includes preset number after modulation (being indicated here with C) a eigenmatrix 304.Here it is indicated with C × H × W, H is characterized the line number of matrix, and W is characterized matrix Columns.Subsequently, electronic equipment 301 include using above-mentioned convolutional neural networks, with the corresponding convolutional layer of characteristic pattern after modulation, First process of convolution is carried out to characteristic pattern after modulation, eigenmatrix 305 after C the first convolution is obtained, can also equally use C here × H × W is indicated.

Then, electronic equipment 301 is from eigenmatrix 305, determining each pixel pair after preset number the first convolution The pixel characteristic vector answered.Wherein, each pixel characteristic vector includes C element.For example, for a pixel, electronics is set Standby 301 from after each first convolution in eigenmatrix, will element extraction corresponding with the pixel come out and group be combined into one to Amount is used as pixel characteristic vector.

Subsequently, electronic equipment 301 is based on obtained pixel characteristic vector, determines pixel relationship matrix 306.For example, Electronic equipment 301 combines pixel characteristic vector, eigenmatrix 307 (size is (H × W) × C) after being combined, then will combination Afterwards eigenmatrix 307 with combine after the transposed matrix 308 (size be C × (H × W)) of eigenmatrix be multiplied, will be obtained after multiplication Matrix be determined as pixel relationship matrix 306 (size be (H × W) × (H × W)).

Finally, electronic equipment 301 is based on pixel relationship matrix, characteristic pattern after modulation is converted, generates spy after transformation Levy Figure 30 9.For example, the corresponding eigenmatrix in each channel that characteristic pattern after modulation includes is converted to the feature vector of N-dimensional, In, N=H × W, then be the matrix 310 of C row N column by obtained combination of eigenvectors, then by the matrix and pixel relationship Matrix 306 is multiplied, and obtains the matrix 311 of C row N column, and the N column element for finally including by every row is converted to H row W column again Eigenmatrix, wherein H is the line number of the element that includes of target image 302, and W is the column for the element that target image 302 includes Number.To finally obtain characteristic pattern 309 after the transformation corresponding to C eigenmatrix.

The method provided by the above embodiment of the disclosure, by obtaining target image, and the feature of determining target image Figure, then the modulation of single order spatial attention, characteristic pattern after modulate, then to characteristic pattern progress the after modulation are carried out to characteristic pattern One process of convolution, obtains eigenmatrix after preset number the first convolution, then from after obtained first convolution in eigenmatrix, It determines the corresponding pixel characteristic vector of pixel that target image includes, then is based on pixel characteristic vector, determine that pixel is closed It is matrix, is finally based on pixel relationship matrix, characteristic pattern after modulation is converted, generates characteristic pattern after transformation.Due to transformation Characteristic pattern is generated according to identified pixel relationship matrix afterwards, and therefore, characteristic pattern can be used for characterizing target figure after transformation The relationship between pixel that picture includes, so that characteristic pattern can more fully characterize target image after obtained transformation Feature helps to improve the accuracy identified to image, and improves from the accurate of extracting target from images object images Property.

With continued reference to Fig. 4, the process of one embodiment of the method for the image for identification according to the disclosure is shown 400.The method of the image for identification, comprising the following steps:

Step 401, images to be recognized is obtained.

In the present embodiment, (such as server shown in FIG. 1 or terminal are set the executing subject of the method for image for identification It is standby) it can be from long-range or from local obtain images to be recognized.Wherein, images to be recognized includes target object image.Target object Image is the image for characterizing target object, and it is signified that target object can be the image that following convolutional neural networks can identify The object shown.As an example, target object image can include but is not limited to following at least one image: facial image, human body Image, animal painting.

Step 402, the convolutional neural networks that images to be recognized input is trained in advance, export for characterizing images to be recognized In position of the target object image in images to be recognized location information and for characterizing class belonging to target object image Other classification information.

In the present embodiment, images to be recognized can be inputted convolutional neural networks trained in advance by above-mentioned executing subject, It exports the location information for characterizing position of the target object image in images to be recognized in images to be recognized and is used for table Levy the classification information of classification belonging to target object image.

Wherein, convolutional neural networks include convolutional layer and classification layer, and convolutional layer is used to execute using images to be recognized above-mentioned The method (that is, using images to be recognized as the target image in Fig. 2 corresponding embodiment) of Fig. 2 corresponding embodiment description, generates and becomes Change rear characteristic pattern.Layer of classifying is used to classify to the pixel that images to be recognized includes based on characteristic pattern after transformation, be generated class Other information and location information.

In general, classification layer may include full articulamentum and classifier, full articulamentum is used for the various spies for generating convolutional layer Sign figure (including characteristic pattern after above-mentioned transformation, it can also include the spy that other methods for not utilizing Fig. 2 corresponding embodiment to describe generate Sign figure) it integrates, generate the feature vector for classification.Classifier can use features described above vector, to above-mentioned to be identified The pixel that image includes is classified, and may thereby determine that the region for belonging to the pixel composition of some classification, which can be with It is characterized using location information, the category can be characterized with classification information.

As an example, location information may include the coordinate value of four angle points of rectangle, each coordinate value is corresponded respectively to A pixel in images to be recognized can determine position of the target object image in images to be recognized according to coordinate value.

Above-mentioned classification information can include but is not limited to the information of following at least one form: text, number, symbol.Example Such as, classification information can be text " face ", be facial image for characterizing target object image.

In practice, above-mentioned executing subject or other electronic equipments can use preset training sample set to initial convolution Neural network is trained, to obtain above-mentioned convolutional neural networks.Specifically, as an example, training sample may include sample This image and the mark classification information and labeling position information that sample image is marked.Execution for training convolutional neural networks Main body can use machine learning method, and the sample image for including using the training sample in training sample set, will as input Mark classification information corresponding with the sample image of input and labeling position information are as desired output, to initial convolution nerve net Network is trained, for the sample image of each training input, available reality output.Wherein, reality output is initial volume The data of product neural network reality output, for characterizing classification information and location information.Then, above-mentioned executing subject can use Gradient descent method and back propagation are based on reality output and desired output, adjust the parameter of initial convolutional neural networks, will be every Initial convolutional neural networks of the convolutional neural networks obtained after secondary adjusting parameter as training next time, and meeting preset instruction In the case where practicing termination condition, terminate training, so that training obtains convolutional neural networks.Above-mentioned preset trained termination condition can To include but is not limited at least one of following: the training time is more than preset duration；Frequency of training is more than preset times；Using default Loss function (such as cross entropy loss function) calculate resulting penalty values and be less than default penalty values threshold value.

Above-mentioned location information and classification information can export in various ways.For example, location information and classification can be believed Breath is shown on the display that above-mentioned executing subject includes；Or it sends location information and classification information to and above-mentioned execution master On the electronic equipment of body communication connection；Or color corresponding with classification information is generated in images to be recognized according to location information Rectangle frame.

The convolutional neural networks that the present embodiment uses, the method due to that can execute the description of Fig. 2 corresponding embodiment, are generated Transformation after characteristic pattern can be used for characterizing the relationship between each pixel in images to be recognized, according to each pixel it Between relationship, can more accurately be classified to the pixel that images to be recognized includes, thus realize more precisely, efficiently Identify image.

In some optional implementations of the present embodiment, above-mentioned executing subject is also based on location information, to Identify extracting target from images object images and display.Specifically, above-mentioned executing subject can determine target according to location information Position of the object images in images to be recognized, so that target object image zooming-out be come out.Target object image can be shown On the display screen that above-mentioned executing subject includes, also it may be displayed on aobvious with the electronic equipment of above-mentioned executing subject communication connection In display screen.This implementation can more accurately be mentioned since above-mentioned convolutional neural networks are utilized from images to be recognized It takes and displaying target object images.

The method provided by the above embodiment of the disclosure executes the corresponding implementation of above-mentioned Fig. 2 by using convolutional neural networks Example description method, images to be recognized is identified, output for characterize the target object image in images to be recognized to The location information of the position in image and the classification information for characterizing classification belonging to target object image are identified, thus effectively The relationship between the pixel that characteristic pattern after transformation characterizes is utilized in ground, and the pixel for including to images to be recognized carries out more acurrate Ground classification, realize more precisely, efficiently identify image.

With further reference to Fig. 5, as the realization to method shown in above-mentioned Fig. 2, present disclose provides one kind for generating spy One embodiment of the device of figure is levied, the Installation practice is corresponding with embodiment of the method shown in Fig. 2, which specifically can be with Applied in various electronic equipments.

As shown in figure 5, the present embodiment includes: first acquisition unit 501 for generating the device 500 of characteristic pattern, matched It is set to acquisition target image, and determines the characteristic pattern of target image, wherein characteristic pattern includes at least one channel, Mei Getong Road corresponds to an eigenmatrix；Modulation unit 502 is configured to carry out the modulation of single order spatial attention to characteristic pattern, obtain Characteristic pattern after modulation；Convolution unit 503 is configured to carry out the first process of convolution to characteristic pattern after modulation, obtains preset number Eigenmatrix after a first convolution, wherein the characteristic pattern obtained after eigenmatrix and the first process of convolution after the first convolution includes Preset number channel correspond；First determination unit 504, is configured in the pixel for including for target image Pixel, from eigenmatrix, determining the corresponding pixel characteristic vector of the pixel after preset number the first convolution；Second really Order member 505 is configured to determine pixel relationship matrix, wherein pixel relationship matrix based on obtained pixel characteristic vector Including element be used to characterize incidence relation between the pixel that target image includes；Converter unit 506 is configured to be based on Pixel relationship matrix converts characteristic pattern after modulation, generates characteristic pattern after transformation.

In the present embodiment, first acquisition unit 501 can be by wired connection mode or radio connection from remote Journey, or target image is obtained from local.Wherein, target image is to handle it, to generate its corresponding characteristic pattern Image.For example, target image can be the camera shooting that above-mentioned apparatus 500 includes image or above-mentioned apparatus 500 from The image extracted in preset image collection.

Above-mentioned first acquisition unit 501 may further determine that the characteristic pattern of target image.Wherein, characteristic pattern (feature Map) for characterizing the feature (such as color characteristic, gray feature etc.) of image.In general, characteristic pattern includes at least one channel, Each channel is used to characterize a kind of feature of image, meanwhile, each channel corresponds to an eigenmatrix, every in eigenmatrix A element, a pixel for including corresponding to target image.

Above-mentioned first acquisition unit 501 can determine the characteristic pattern of target image in various manners.As an example, above-mentioned The color value (including R value, G value, B value) for each pixel that first acquisition unit 501 can include according to target image generates Characteristic pattern including three channels (the respectively channel R, the channel G, channel B), each channel correspond to an eigenmatrix, wherein Element be corresponding color color value.

For another example target image can be inputted preset convolutional neural networks, convolution mind by above-mentioned first acquisition unit 501 The convolutional layer for including through network can extract the feature of target image, generate characteristic pattern.In general, convolutional layer may include at least one A convolution kernel, each convolution kernel can be used for generating an eigenmatrix.It should be noted that in general, convolutional neural networks can To include multiple convolutional layers, characteristic pattern used in the present embodiment can be the characteristic pattern that any convolutional layer generates.

In the present embodiment, modulation unit 502 can carry out the modulation of single order spatial attention to characteristic pattern, after obtaining modulation Characteristic pattern.Wherein, single order spatial attention modulation (Spatial Attention) refer to, by characteristic pattern it is corresponding at least one Eigenmatrix, is remapped to multiple vectors, and each vector therein corresponds to the pixel that target image includes, then benefit With preset function, operation (such as the operations such as weighting, classification, convolution) are carried out to these vectors, finally convert operation result It is new characteristic pattern as characteristic pattern after modulation, wherein characteristic pattern includes at least one channel after modulation, and each channel corresponds to One eigenmatrix.Since above-mentioned operation is to carry out linear operation, referred to as single order space transforms to above-mentioned multiple vectors Power modulation.

In the present embodiment, convolution unit 503 can carry out the first process of convolution to characteristic pattern after modulation, obtain present count Eigenmatrix after mesh the first convolution, wherein the characteristic pattern packet obtained after eigenmatrix and the first process of convolution after the first convolution The preset number channel included corresponds.In general, above-mentioned convolution unit 503 can use preset preset number convolution kernel At least one eigenmatrix corresponding to characteristic pattern after modulation carries out the first process of convolution, to obtain the preset number first volume Eigenmatrix after product.Convolution kernel is usually the form of matrix, and element therein is preset weighted value, can be with using weighted value At least one eigenmatrix corresponding to characteristic pattern after modulation carries out convolution algorithm.It should be noted that the power that convolution kernel includes Weight values can be pre-set, be also possible to advance with machine learning method, to convolutional neural networks belonging to convolution kernel Determined by after being trained.

In the present embodiment, the pixel in pixel for including for target image, above-mentioned first determination unit 504 can With from eigenmatrix, determining the corresponding pixel characteristic vector of the pixel after preset number the first convolution.

In general, the element after each first convolution in eigenmatrix, the pixel for including with target image is corresponded.It is right In a pixel, above-mentioned first determination unit 504 can be from after each first convolution in eigenmatrix, will be with the pixel pair The element extraction answered comes out and group is combined into a vector as pixel characteristic vector.As an example it is supposed that after having C the first convolution Eigenmatrix, eigenmatrix is the matrix of H row W column after each first convolution.Wherein, H is the pixel that target image includes Line number, W are the columns for the pixel that target image includes.The then pixel characteristic vector of available N number of C dimension, wherein N=H × W。

In the present embodiment, the second determination unit 505 can be based on obtained pixel characteristic vector, determine pixel relationship Matrix.Wherein, the element that pixel relationship matrix includes is used to characterize the incidence relation (example between the pixel that target image includes Such as the relationship between color, gray scale, brightness).

Specifically, as an example, above-mentioned second determination unit 505 can combine pixel characteristic vector, after obtaining combination Eigenmatrix, then by eigenmatrix after combination with combine after eigenmatrix transposed matrix be multiplied, the matrix that will be obtained after multiplication It is determined as pixel relationship matrix.

In the present embodiment, converter unit 506 can be based on pixel relationship matrix, convert to characteristic pattern after modulation, Generate characteristic pattern after converting.Characteristic pattern can be used for characterizing the pass of the association between each pixel that target image includes after transformation System.

In some optional implementations of the present embodiment, characteristic pattern includes preset number channel after modulation, each Channel corresponds to an eigenmatrix；And converter unit 506 may include: the second conversion module (not shown), be matched It is set to the channel in the preset number channel for including for characteristic pattern after modulating, the corresponding eigenmatrix in the channel is converted to Second channel feature vector；Third composite module (not shown) is configured to obtained second channel feature vector Combination obtains matrix after third combination；Second generation module (not shown), matrix and picture after being configured to combine third Plain relational matrix is multiplied, and characteristic pattern after converting is generated based on obtained matrix after being multiplied.

In some optional implementations of the present embodiment, the second determination unit 505 may include: the first composite module (not shown) is configured to combine obtained pixel characteristic vector, obtains matrix after the first combination；Convolution module (not shown) is configured to carry out the second process of convolution to characteristic pattern after modulation, after obtaining the second convolution of preset number Eigenmatrix, wherein the preset number that the characteristic pattern obtained after eigenmatrix and the second process of convolution after the second convolution includes Channel corresponds；First conversion module (not shown) is configured to for the second convolution of obtained preset number Eigenmatrix after the second convolution in eigenmatrix afterwards, by eigenmatrix after second convolution be converted to first passage feature to Amount；Second composite module (not shown) is configured to obtained first passage combination of eigenvectors obtaining second group Matrix after conjunction；First generation module (not shown), matrix phase after being configured to combine matrix after the first combination with second Multiply, pixel relationship matrix is generated based on the matrix obtained after multiplication.

In some optional implementations of the present embodiment, the first generation module can be further configured to: to phase The element that the matrix obtained after multiplying includes is normalized, and obtains pixel relationship matrix.

The device provided by the above embodiment 500 of the disclosure, by obtaining target image, and the spy of determining target image Sign figure, then the modulation of single order spatial attention is carried out to characteristic pattern, then characteristic pattern after being modulated carries out characteristic pattern after modulation First process of convolution, obtains eigenmatrix after preset number the first convolution, then from eigenmatrix after obtained first convolution In, it determines the corresponding pixel characteristic vector of pixel that target image includes, then be based on pixel characteristic vector, determines pixel Relational matrix is finally based on pixel relationship matrix, converts to characteristic pattern after modulation, generates characteristic pattern after transformation.Due to becoming Changing rear characteristic pattern is generated according to identified pixel relationship matrix, and therefore, characteristic pattern can be used for characterizing target after transformation The relationship between pixel that image includes, so that characteristic pattern can more fully characterize target image after obtained transformation Feature, help to improve the accuracy identified to image, and improve the standard from extracting target from images object images True property.

With further reference to Fig. 6, as the realization to method shown in above-mentioned Fig. 4, present disclose provides one kind to scheme for identification One embodiment of the device of picture, the Installation practice is corresponding with embodiment of the method shown in Fig. 4, which can specifically answer For in various electronic equipments.

As shown in fig. 6, the device 600 of the image for identification of the present embodiment includes: second acquisition unit 601, it is configured At acquisition images to be recognized, wherein images to be recognized includes target object image；Output unit 602, being configured to will be to be identified Image input convolutional neural networks trained in advance, export for characterizing the target object image in images to be recognized to be identified The location information of position in image and classification information for characterizing classification belonging to target object image, wherein convolution mind It include convolutional layer and classification layer through network, convolutional layer is used to execute above-mentioned Fig. 2 corresponding embodiment description using images to be recognized Method, generates characteristic pattern after transformation, and classification layer is used to carry out the pixel that images to be recognized includes based on characteristic pattern after transformation Classification generates classification information and location information.

In the present embodiment, second acquisition unit 601 can be from long-range or from local obtain images to be recognized.Wherein, to Identify that image includes target object image.Target object image is the image for characterizing target object, and target object can be Object indicated by the image that following convolutional neural networks can identify.As an example, target object image may include but not It is limited to following at least one image: facial image, human body image, animal painting.

In the present embodiment, images to be recognized can be inputted convolutional neural networks trained in advance by output unit 602, It exports the location information for characterizing position of the target object image in images to be recognized in images to be recognized and is used for table Levy the classification information of classification belonging to target object image.

Above-mentioned location information and classification information can export in various ways.For example, location information and classification can be believed Breath is shown on the display that above-mentioned apparatus 600 includes；Or it sends location information and classification information to and above-mentioned apparatus 600 On the electronic equipment of communication connection；Or color corresponding with classification information is generated in images to be recognized according to location information Rectangle frame.

In some optional implementations of the present embodiment, the device 600 can also include: display unit (in figure not Show), it is configured to extract target object image and display from images to be recognized based on location information.

It is corresponding to execute above-mentioned Fig. 2 by using convolutional neural networks for the device provided by the above embodiment 600 of the disclosure The method of embodiment description, identifies images to be recognized, exports for characterizing the target object image in images to be recognized The location information of position in images to be recognized and classification information for characterizing classification belonging to target object image, thus The relationship between the pixel that characteristic pattern characterizes after converting is efficiently utilized, the pixel that images to be recognized includes is carried out more Accurately classify, realize more precisely, efficiently identify image.

Below with reference to Fig. 7, it illustrates the electronic equipment that is suitable for being used to realize embodiment of the disclosure, (example is as shown in figure 1 Server or terminal device) 700 structural schematic diagram.Terminal device in embodiment of the disclosure can include but is not limited to all As mobile phone, laptop, digit broadcasting receiver, PDA (personal digital assistant), PAD (tablet computer), PMP are (portable Formula multimedia player), the mobile terminal and such as number TV, desk-top meter of car-mounted terminal (such as vehicle mounted guidance terminal) etc. The fixed terminal of calculation machine etc..Electronic equipment shown in Fig. 7 is only an example, should not be to the function of embodiment of the disclosure Any restrictions are brought with use scope.

As shown in fig. 7, electronic equipment 700 may include processing unit (such as central processing unit, graphics processor etc.) 701, random access can be loaded into according to the program being stored in read-only memory (ROM) 702 or from storage device 708 Program in memory (RAM) 703 and execute various movements appropriate and processing.In RAM 703, it is also stored with electronic equipment Various programs and data needed for 700 operations.Processing unit 701, ROM 702 and RAM 703 pass through the phase each other of bus 704 Even.Input/output (I/O) interface 705 is also connected to bus 704.

In general, following device can connect to I/O interface 705: including such as touch screen, touch tablet, keyboard, mouse, taking the photograph As the input unit 706 of head, microphone, accelerometer, gyroscope etc.；Including such as liquid crystal display (LCD), loudspeaker, vibration The output device 707 of dynamic device etc.；Storage device 708 including such as tape, hard disk etc.；And communication device 709.Communication device 709, which can permit electronic equipment 700, is wirelessly or non-wirelessly communicated with other equipment to exchange data.Although Fig. 7 shows tool There is the electronic equipment 700 of various devices, it should be understood that being not required for implementing or having all devices shown.It can be with Alternatively implement or have more or fewer devices.Each box shown in Fig. 7 can represent a device, can also root According to needing to represent multiple devices.

Particularly, in accordance with an embodiment of the present disclosure, it may be implemented as computer above with reference to the process of flow chart description Software program.For example, embodiment of the disclosure includes a kind of computer program product comprising be carried on computer-readable medium On computer program, which includes the program code for method shown in execution flow chart.In such reality It applies in example, which can be downloaded and installed from network by communication device 709, or from storage device 708 It is mounted, or is mounted from ROM 702.When the computer program is executed by processing unit 701, the implementation of the disclosure is executed The above-mentioned function of being limited in the method for example.

It is situated between it should be noted that computer-readable medium described in embodiment of the disclosure can be computer-readable signal Matter or computer-readable medium either the two any combination.Computer-readable medium for example can be --- but not Be limited to --- electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor system, device or device, or any above combination.Meter The more specific example of calculation machine readable medium can include but is not limited to: have the electrical connection, portable of one or more conducting wires Computer disk, hard disk, random access storage device (RAM), read-only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), light storage device, magnetic memory device or The above-mentioned any appropriate combination of person.

In embodiment of the disclosure, computer-readable medium can be any tangible medium for including or store program, The program can be commanded execution system, device or device use or in connection.And in embodiment of the disclosure In, computer-readable signal media may include in a base band or as the data-signal that carrier wave a part is propagated, wherein holding Computer-readable program code is carried.The data-signal of this propagation can take various forms, including but not limited to electromagnetism Signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be computer-readable medium with Outer any computer-readable medium, the computer-readable signal media can be sent, propagated or transmitted for being held by instruction Row system, device or device use or program in connection.The program code for including on computer-readable medium It can transmit with any suitable medium, including but not limited to: electric wire, optical cable, RF (radio frequency) etc. or above-mentioned any conjunction Suitable combination.

Above-mentioned computer-readable medium can be included in above-mentioned electronic equipment；It is also possible to individualism, and not It is fitted into the electronic equipment.Above-mentioned computer-readable medium carries one or more program, when said one or more When a program is executed by the electronic equipment, so that the electronic equipment: obtaining target image, and determine the feature of target image Figure, wherein the characteristic pattern includes at least one channel, and each channel corresponds to an eigenmatrix；One is carried out to characteristic pattern The modulation of rank spatial attention, characteristic pattern after being modulated；First process of convolution is carried out to characteristic pattern after modulation, obtains preset number Eigenmatrix after a first convolution, wherein the characteristic pattern obtained after eigenmatrix and the first process of convolution after the first convolution includes Preset number channel correspond；The pixel in pixel for including for target image, from preset number first After convolution in eigenmatrix, the corresponding pixel characteristic vector of the pixel is determined；Based on obtained pixel characteristic vector, determine Pixel relationship matrix, wherein the element that pixel relationship matrix includes is used to characterize the pass between the pixel that target image includes Connection relationship；Based on pixel relationship matrix, characteristic pattern after modulation is converted, generates characteristic pattern after transformation.

In addition, when said one or multiple programs are executed by the electronic equipment, it is also possible that the electronic equipment: obtaining Take images to be recognized, wherein images to be recognized includes target object image；By images to be recognized input convolution mind trained in advance Through network, export location information for characterizing position of the target object image in images to be recognized in images to be recognized and For characterizing the classification information of classification belonging to target object image.

The behaviour for executing embodiment of the disclosure can be write with one or more programming languages or combinations thereof The computer program code of work, described program design language include object oriented program language-such as Java, Smalltalk, C++ further include conventional procedural programming language-such as " C " language or similar program design language Speech.Program code can be executed fully on the user computer, partly be executed on the user computer, as an independence Software package execute, part on the user computer part execute on the remote computer or completely in remote computer or It is executed on server.In situations involving remote computers, remote computer can pass through the network of any kind --- packet It includes local area network (LAN) or wide area network (WAN)-is connected to subscriber computer, or, it may be connected to outer computer (such as benefit It is connected with ISP by internet).

Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation A part of one module, program segment or code of table, a part of the module, program segment or code include one or more use The executable instruction of the logic function as defined in realizing.It should also be noted that in some implementations as replacements, being marked in box The function of note can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are actually It can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it to infuse Meaning, the combination of each box in block diagram and or flow chart and the box in block diagram and or flow chart can be with holding The dedicated hardware based system of functions or operations as defined in row is realized, or can use specialized hardware and computer instruction Combination realize.

Being described in unit involved in embodiment of the disclosure can be realized by way of software, can also be passed through The mode of hardware is realized.Described unit also can be set in the processor, for example, can be described as: a kind of processor Including first acquisition unit, modulation unit, convolution unit, the first determination unit, the second determination unit and converter unit.Wherein, The title of these units does not constitute the restriction to the unit itself under certain conditions, for example, first acquisition unit can be with It is described as " obtain target image, and determine the unit of the characteristic pattern of target image ".

Above description is only the preferred embodiment of the disclosure and the explanation to institute's application technology principle.Those skilled in the art Member it should be appreciated that embodiment of the disclosure involved in invention scope, however it is not limited to the specific combination of above-mentioned technical characteristic and At technical solution, while should also cover do not depart from foregoing invention design in the case where, by above-mentioned technical characteristic or its be equal Feature carries out any combination and other technical solutions for being formed.Such as disclosed in features described above and embodiment of the disclosure (but It is not limited to) technical characteristic with similar functions is replaced mutually and the technical solution that is formed.

Claims

1. a kind of method for generating characteristic pattern, comprising:

Target image is obtained, and determines the characteristic pattern of the target image, wherein the characteristic pattern includes that at least one is logical Road, each channel correspond to an eigenmatrix；

The modulation of single order spatial attention, characteristic pattern after being modulated are carried out to the characteristic pattern；

First process of convolution is carried out to characteristic pattern after the modulation, obtains eigenmatrix after the first convolution of preset number, wherein The preset number channel that the characteristic pattern obtained after eigenmatrix and the first process of convolution after first convolution includes corresponds；

The pixel in pixel for including for the target image, from eigenmatrix after the first convolution of the preset number In, determine the corresponding pixel characteristic vector of the pixel；

Based on obtained pixel characteristic vector, pixel relationship matrix is determined, wherein the element that the pixel relationship matrix includes For characterizing the incidence relation between the pixel that the target image includes；

Based on the pixel relationship matrix, characteristic pattern after the modulation is converted, generates characteristic pattern after transformation.

2. it is described to be based on obtained pixel characteristic vector according to the method described in claim 1, wherein, determine pixel relationship Matrix, comprising:

Obtained pixel characteristic vector is combined, matrix after the first combination is obtained；

Second process of convolution is carried out to characteristic pattern after the modulation, obtains eigenmatrix after the second convolution of preset number, wherein The preset number channel that the characteristic pattern obtained after eigenmatrix and the second process of convolution after second convolution includes corresponds；

For eigenmatrix after the second convolution in eigenmatrix after the second convolution of obtained preset number, by the volume Two Eigenmatrix is converted to first passage feature vector after product；

By obtained first passage combination of eigenvectors, matrix after the second combination is obtained；

Matrix multiple after combining matrix after first combination with described second, generates pixel based on the matrix obtained after multiplication Relational matrix.

3. it is described that pixel relationship matrix is generated based on the matrix obtained after multiplication according to the method described in claim 2, wherein, Include:

The element for including to the matrix obtained after multiplication is normalized, and obtains pixel relationship matrix.

4. method described in one of -3 according to claim 1, wherein characteristic pattern includes preset number channel after the modulation, Each channel corresponds to an eigenmatrix；And

It is described to be based on the pixel relationship matrix, characteristic pattern after the modulation is converted, characteristic pattern after transformation, packet are generated It includes:

The channel in preset number channel for including for characteristic pattern after the modulation turns the corresponding eigenmatrix in the channel It is changed to second channel feature vector；

By obtained second channel combination of eigenvectors, matrix after third combination is obtained；

By matrix and the pixel relationship matrix multiple after third combination, transformation is generated based on obtained matrix after being multiplied Characteristic pattern afterwards.

5. a kind of method of image for identification, comprising:

Obtain images to be recognized, wherein the images to be recognized includes target object image；

Images to be recognized input convolutional neural networks trained in advance are exported for characterizing in the images to be recognized Target object image is in the location information of the position in the images to be recognized and for characterizing belonging to the target object image Classification classification information, wherein the convolutional neural networks include convolutional layer and classification layer, the convolutional layer be used for utilize institute It states images to be recognized perform claim and requires method described in one of 1-4, generate characteristic pattern after transformation, classification layer is used for based on described Characteristic pattern after transformation, the pixel for including to the images to be recognized are classified, and classification information and location information are generated.

6. according to the method described in claim 5, wherein, the method also includes:

Based on the location information, target object image and display are extracted from the images to be recognized.

7. a kind of for generating the device of characteristic pattern, comprising:

First acquisition unit is configured to obtain target image, and determines the characteristic pattern of the target image, wherein described Characteristic pattern includes at least one channel, and each channel corresponds to an eigenmatrix；

Modulation unit is configured to carry out the modulation of single order spatial attention, characteristic pattern after being modulated to the characteristic pattern；

Convolution unit is configured to carry out the first process of convolution to characteristic pattern after the modulation, obtains the preset number first volume Eigenmatrix after product, wherein the present count that the characteristic pattern obtained after eigenmatrix and the first process of convolution after the first convolution includes Mesh channel corresponds；

First determination unit, the pixel being configured in the pixel for including for the target image, from the present count After the first convolution of mesh in eigenmatrix, the corresponding pixel characteristic vector of the pixel is determined；

Second determination unit is configured to determine pixel relationship matrix, wherein described based on obtained pixel characteristic vector The element that pixel relationship matrix includes is used to characterize the incidence relation between the pixel that the target image includes；

Converter unit is configured to convert characteristic pattern after the modulation based on the pixel relationship matrix, generates transformation Characteristic pattern afterwards.

8. device according to claim 7, wherein second determination unit includes:

First composite module is configured to combine obtained pixel characteristic vector, obtains matrix after the first combination；

Convolution module is configured to carry out the second process of convolution to characteristic pattern after the modulation, obtains preset number volume Two Eigenmatrix after product, wherein the present count that the characteristic pattern obtained after eigenmatrix and the second process of convolution after the second convolution includes Mesh channel corresponds；

First conversion module is configured to for the second convolution after the second convolution of obtained preset number in eigenmatrix Eigenmatrix after second convolution is converted to first passage feature vector by eigenmatrix afterwards；

Second composite module is configured to obtained first passage combination of eigenvectors obtaining matrix after the second combination；

First generation module, matrix multiple after being configured to combine matrix after first combination with described second, is based on phase The matrix obtained after multiplying generates pixel relationship matrix.

9. device according to claim 8, wherein first generation module is further configured to:

10. device according to claim 7, wherein characteristic pattern includes preset number channel, Mei Getong after the modulation Road corresponds to an eigenmatrix；And

The converter unit includes:

Second conversion module is configured to the channel in the preset number channel for including for characteristic pattern after the modulation, will The corresponding eigenmatrix in the channel is converted to second channel feature vector；

Third composite module is configured to obtained second channel combination of eigenvectors obtaining matrix after third combination；

Second generation module, matrix and the pixel relationship matrix multiple after being configured to combine the third, based on multiplication Obtained matrix generates characteristic pattern after transformation afterwards.

11. a kind of device of image for identification, comprising:

Second acquisition unit is configured to obtain images to be recognized, wherein the images to be recognized includes target object image；

Output unit is configured to inputting the images to be recognized into convolutional neural networks trained in advance, export for characterizing The location information of position of the target object image in the images to be recognized in the images to be recognized and for characterizing State the classification information of classification belonging to target object image, wherein the convolutional neural networks include convolutional layer and classification layer, institute Convolutional layer is stated for requiring method described in one of 1-4 using the images to be recognized perform claim, generates characteristic pattern after converting, Layer of classifying is used for based on characteristic pattern after the transformation, and the pixel for including to the images to be recognized is classified, and generates classification Information and location information.

12. device according to claim 11, wherein described device further include:

Display unit, is configured to based on the location information, and target object image is extracted from the images to be recognized and is shown Show.

13. a kind of electronic equipment, comprising:

One or more processors；

Storage device is stored thereon with one or more programs,

When one or more of programs are executed by one or more of processors, so that one or more of processors are real Now such as method as claimed in any one of claims 1 to 6.

14. a kind of computer-readable medium, is stored thereon with computer program, wherein the realization when program is executed by processor Such as method as claimed in any one of claims 1 to 6.