CN111340124A - Method and device for identifying entity category in image - Google Patents

Abstract

The present application provides a method and device for identifying entity categories in an image, wherein the method includes: extracting image features of a target image according to a preset algorithm, and determining at least one target area in the target image including the target entity according to the image features; extracting at least one target area The classification features of each target area in a target area, and fuse all the classification features to obtain the local classification features; extract the global classification features of the target image, and fuse the local classification features and global classification features to obtain the target classification features; determine the target according to the target classification features The entity class of the entity. As a result, the local features of the entity's existence are strengthened, and the entity category in the image can be accurately determined according to the combination of the global feature and the local feature, which solves the technical problem in the prior art that it is difficult to determine the specific category of the entity due to the similarity of the global features of the image.

Description

Method and device for identifying entity categories in images

technical field

The present application relates to the technical field of image processing, and in particular, to a method and apparatus for identifying entity categories in an image.

Background technique

As a basic and important task of image classification and understanding, image object classification and understanding has aroused great research interest in recent years, and has been successfully deployed in many application products, intelligently solving many practical problems. With the rapid development of deep learning technology in recent years, deep learning has also become the most advanced technology in image object classification. That is, learning entity categories in an image based on the global depth features of the image.

In the related art, the global features in images are similar in many scenarios, and it is difficult to identify entity categories according to the global features. There are entities of different categories into one category. For example, when identifying the category of food, due to the The high similarity increases the difficulty in determining the category of food.

SUMMARY OF THE INVENTION

The present application aims to solve one of the technical problems in the related art at least to a certain extent.

To this end, the first purpose of this application is to propose a method for identifying entity categories in images, so as to strengthen the local features of the entity's existence. According to the combination of global features and local features, the entity categories in the image can be accurately determined. In the prior art, it is difficult to determine the specific category of the entity due to the similarity of the global features of the images.

The second objective of the present application is to provide a device for identifying the type of entity in the image.

The third objective of the present application is to provide a terminal device.

A fourth object of the present application is to propose a computer-readable storage medium.

In order to achieve the above purpose, a method for identifying entity categories in an image proposed by an embodiment of the first aspect of the present application includes: extracting image features of a target image according to a preset algorithm, and determining in the target image according to the image features including at least one target area of the target entity; extracting the classification features of each target area in the at least one target area, and fusing all the classification features to obtain local classification features; extracting the global classification features of the target image, and fusing the local classification features The classification feature and the global classification feature obtain a target classification feature; and the entity category of the target entity is determined according to the target classification feature.

The device for identifying entity categories in an image proposed by an embodiment of the second aspect of the present application includes: a first determination module, configured to extract image features of a target image according to a preset algorithm, and determine, according to the image features, the at least one target area of the target entity; the first fusion module is used to extract the classification features of each target area in the at least one target area, and fuse all classification features to obtain local classification features; the second fusion module is used to extract all the classification features. The global classification feature of the target image is obtained, and the local classification feature and the global classification feature are combined to obtain the target classification feature; the second determination module is used for determining the entity category of the target entity according to the target classification feature.

The terminal device proposed by the embodiment of the third aspect of the present application includes: a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the present invention when the processor executes the computer program. The method for identifying entity categories in images described in the embodiments of the first aspect of the application is provided.

The computer-readable storage medium provided by the embodiment of the fourth aspect of the present application stores a computer program thereon, and when the computer program is executed by the processor, implements the method for identifying the entity category in the image according to the embodiment of the first aspect of the present application.

The technical solution provided by this application at least includes the following technical effects:

Extract the image features of the target image according to the preset algorithm, determine at least one target region containing the target entity in the target image according to the image features, extract the classification features of each target region in the at least one target region, and fuse all the classification features to obtain local classification Then, the global classification feature of the target image is extracted, and the local classification feature and the global classification feature are combined to obtain the target classification feature. Finally, the entity category of the target entity is determined according to the target classification feature. Thus, the entity category of the target entity is determined according to the target classification feature.

Additional aspects and advantages of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a schematic flowchart of a method for identifying an entity type provided by an embodiment of the present application;

FIG. 2 is a schematic diagram of an application scenario for identifying an entity type provided by an embodiment of the present application;

3 is a schematic diagram of the architecture of the P-net algorithm according to an embodiment of the present application;

4 is a schematic diagram of a P-net algorithm training process according to an embodiment of the present application;

5 is an architectural diagram of an R-Net algorithm according to an embodiment of the present application;

6 is a schematic flowchart of a method for acquiring a target area according to a specific embodiment of the present application;

7 is a schematic diagram of a training flow of the R-Net algorithm according to an embodiment of the present application;

FIG. 8 is a schematic diagram of the architecture of a subdivision model according to an embodiment of the present application;

FIG. 9 is a schematic diagram of training of a subdivision model according to an embodiment of the present application;

10 is a schematic flowchart of a method for identifying an entity category according to another embodiment of the present application;

11 is a schematic diagram of a training process of a classifier according to an embodiment of the present application;

FIG. 12 is a schematic structural diagram of a device for identifying an entity category according to an embodiment of the present application; and

FIG. 13 is a schematic structural diagram of a terminal device according to an embodiment of the present application.

Detailed ways

The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

The method and apparatus for identifying entity categories according to the embodiments of the present application are described below with reference to the accompanying drawings.

The entities in the embodiments of the present application may be food, sporting goods, jewelry, etc., and the identification of categories referred to in the present application may be understood as identification of subcategories under the entity.

FIG. 1 is a schematic flowchart of a method for identifying an entity type according to an embodiment of the present application.

Aiming at the problem of difficulty in distinguishing entity categories mentioned in the above background, an embodiment of the present application provides a method for identifying entity categories to achieve accurate identification of entity categories. As shown in FIG. 1 , the method includes the following steps:

Step 101 , extract image features of the target image according to a preset algorithm, and determine at least one target area including the target entity in the target image according to the image features.

Among them, the target image can be uploaded by the user actively, or it can be the currently captured image automatically determined in the album

In this embodiment, the image feature of the target image is extracted according to a preset algorithm, and at least one target area including the target entity is determined in the target image according to the image feature, that is, the area where the entity exists in the target image is framed. For example, As shown in FIG. 2 , at least one target area can be determined according to the image features, and each target area contains a corresponding entity.

It should be noted that, in different application scenarios, the methods of obtaining at least one target area are different. Examples are as follows:

Example one:

In this example, according to the preset first convolution algorithm, the first one-dimensional feature map of the target image is obtained, that is, the one-dimensional feature map of the extracted image is extracted, so as to more efficiently determine the entities in the image, and then the first one-dimensional feature map is extracted. The image features of the one-dimensional feature map determine at least one candidate region where the target entity is located, and determine the at least one candidate region as at least one target region.

Among them, the extraction methods of the first one-dimensional feature map can be various. As an example, the first convolution algorithm is the P-net algorithm. Referring to the architecture of the P-net algorithm shown in FIG. 3, in practice During the execution process, the target image is processed into a size of 12*12 according to the down-sampling method, and further, the target image is processed into a structure of 12*12*3 according to the color channel.

First, input the target image of 12*12*3 into the first convolutional layer, pass 10 3*3*3 convolution kernels of the first convolutional layer, 2*2 Max Pooling (stride=2 ) operation to generate 10 feature maps of 5*5, and then input the feature map to the second convolutional layer, and generate 16 convolution kernels of 3*3*10 through the second convolutional layer. A 3*3 feature map, input the feature map to the third convolutional layer, and generate 32 1*1 features through 32 3*3*16 convolution kernels of the third convolutional layer Figure, for 32 1*1 feature maps, two 1*1*32 convolution kernels can be used to generate two first one-dimensional 1*1 feature maps for classification. In this example, you can pass Seven 1*1*32 convolution kernels are used to generate nine 1*1 feature maps for regression box judgment to optimize the structure of the algorithm.

Of course, in order to improve the efficacy of the P-net algorithm, as shown in Figure 4, the architecture of the P-net algorithm is pre-trained, and the P-net algorithm is trained after downsampling a large number of sample images. The difference of the marked area calculates the loss function, and optimizes the architecture of the P-net algorithm according to the loss function.

Considering that the P-net algorithm can only roughly determine the region where the corresponding entity is located, in order to further determine the region where the entity is located, the target region can be further refined on the basis of the above candidate regions.

That is, according to the preset second convolution algorithm, the second one-dimensional feature map of each candidate region in the at least one candidate region is extracted, and then the image features of the first one-dimensional feature map are extracted. The image feature converges at least one candidate region, and the converged at least one candidate region is determined as at least one target region. Convergence here can be understood as the deletion of the wrong candidate region of non-existent entities, or the deletion of sub-regions of the candidate region of existing entities that do not contain entities.

In some possible examples, when the second convolution algorithm is the R-Net algorithm, referring to the architecture diagram of the R-Net algorithm shown in FIG. 5 and combining the architecture diagram shown in FIG. 3, the above P-net algorithm is obtained. The candidate area of is processed as an image of 24*24 size, and the image is input into the first convolutional layer of the R-Net algorithm, through 28 3*3*3 convolution After max pooling, 28 feature maps of 11*11 are generated, and the feature map is input to the second convolutional layer, through 48 3*3*28 convolution kernels of the second convolutional layer and 3*3 (stride= 2) After the max pooling, 48 4*4 feature maps are generated, and the feature map is input into the third convolutional layer of the algorithm, and after passing through the 64 2*2*48 convolution kernels of the third convolutional layer , generate 64 3*3 feature maps, convert the 3*3*64 feature map into a 128-size fully connected layer, and then convert the fully connected layer for the regression box classification problem. The fully connected layer is for bounding The fully connected layer of the box position regression problem determines the target area in the candidate area according to the output of the fully connected layer.

That is to say, when the target entity is food, as shown in Figure 6, the target image is input to the PNet algorithm to obtain a large number of candidate regions that may be food, and then converged by Rnet to further filter out unqualified candidates. area. At the same time of filtering, a more refined and compact target area is obtained (because the area obtained by PNet may be too large or incorrectly not contain food). Based on the candidate region (Bounding box, bbox) information predicted by P-Net, R-Net crops the original target image, adjusts the size of the candidate region to a fixed scale, and performs a non-maximum suppression algorithm on the candidate region (NonMaximum Suppression, NMS) screening to identify target regions.

Similarly, referring to Figure 7, it is necessary to pre-train the R-Net algorithm, downsample the image of the target area where the entity is located to a fixed size, and then input it into the R-Net algorithm. According to the R-Net algorithm, the area where the entity is located and The pre-labeled regions are compared, the loss function is determined, and the architecture of the R-Net algorithm is adjusted according to the loss function.

Example two:

In this example, the general image feature of the target entity is preset, and the connected area including the general image feature is determined as the target area.

Step 102: Extract the classification features of each target area in the at least one target area, and fuse all the classification features to obtain local classification features.

Specifically, the classification features of each target region in at least one target region are extracted, and all the classification features are fused to obtain local classification features, that is, the entity category is determined by focusing on the local classification features where the entity is located.

As a possible implementation manner, a subdivision classification model is preset, and the subdivision classification model can extract classification features of each target region in each target region.

In an embodiment of the present application, the screened target areas are sorted. The purpose of sorting is to ensure the consistency of the order. For example, there are only three target areas, which are 4x4, 3x3, and 2x2 in size, and they are subdivided. The length of the sign vector extracted by the model is the same, such as 1x1028, but the actual meaning of each set of vectors is different. Therefore, for example, if 100 random boxes are used for training and the training is sorted from large to small, after the training is completed, when processing the target image, the sorting rules should also be followed, 100 random boxes and the order from large to small. , and then input the target regions into the sub-classification network in turn to obtain the candidate classification features corresponding to each target region. If there are _n target regions corresponding to the vector i, i∈{i ₁ ,...,in } input sub-classification After the model, n eigenvectors v, v∈{v ₁ ,...,v _n } can be obtained.

That is, determine the size of each target area, sort at least one target area according to the size, input at least one target area into the pre-trained sub-classification model in turn according to the sorting result, and determine the classification features of each target area to To ensure the accuracy of the classification feature acquisition, to ensure that the model can be more and more optimized, after each classification feature is extracted, the model is optimized according to the regression mechanism to ensure that the classification features obtained by the subsequent input are more and more accurate.

In an embodiment of the present application, the confidence level of each target area including the target entity can also be determined. For example, the confidence level is determined according to the degree of coincidence with the feature of the target entity. After extracting the candidate classification features of each target area, according to The candidate classification feature and the confidence level of each target area determine the corresponding classification feature, for example, determine the weight value corresponding to the confidence level, and determine the classification feature according to the product of the weight value and the corresponding classification feature.

Step 103 , extract the global classification feature of the target image, and obtain the target classification feature by fusing the local classification feature and the global classification feature.

Specifically, considering the global features of the image, the global classification features of the target image are extracted, and the local classification features and the global classification features are fused to obtain the target classification features. In an embodiment of the present application, global classification features and local classification features may be processed into one-dimensional features and then spliced into target classification features to reduce the amount of computation.

In an embodiment of the present application, referring to FIG. 8 , for the sub-classification model, a sub-classification model based on self-supervised training is adopted. The architecture is divided into two branches. It is worth noting that the feature extractors in the upper and lower branches share a network parameter and structure. The network selection is based on task requirements. For example, ResNet can be selected on the server, and Mobilenet can be selected on devices such as mobile phones. Generally, models pre-trained under large data sets such as Imagnet are used. The upper branch in the image is a traditional forward network. The entire target image is input into the feature extractor to obtain the global classification features, and then sent to the feature fusion network. The guiding network in Figure 8 solves the problem of selecting the candidate frame of the local area, which solves the problem of artificially not knowing which part in the target image is the best. Among them, referring to Figure 9, the guiding network needs to be trained, and the teacher network The purpose is to supervise and guide the feature extraction by comparing the information confidence of each target area obtained by the guiding network. The teacher network adjusts the network parameters of the guiding network according to the loss calculation. After inputting the target area into the guiding network, That is, the lower branch in Figure 8 can obtain the classification features of each target area (in the figure, there are three target areas as an example), and splicing the one-dimensional features extracted from each target area to form a fused one-dimensional vector. The reason for using a one-dimensional vector is that the one-dimensional vector has a small amount of calculation and high processing efficiency, and at the same time, the results obtained are similar to those of a higher-dimensional feature vector.

Step 104: Determine the entity category of the target entity according to the target classification feature.

Specifically, the target classification feature not only reflects local features but also includes global features, and the entity category of the target entity is determined according to the target classification feature, for example, specific subcategories of food are determined.

Further, after the entity category is determined, the albums can be divided according to the category. When there are multiple entity categories, multiple entity categories can be added as image descriptions in the file home, so that users can know without viewing the images. Image type.

In an embodiment of the present application, after the entity category is determined, the corresponding beautification parameters are matched according to the entity category, and appropriate beautification processing is performed on different entities in the target image, which greatly improves the beautification experience.

Referring to FIG. 10 , after the subdivided classification model obtains the fused target classification features, the target classification features can be input into a pre-trained classifier, and the classifier can be a nonlinear classifier, such as a nonlinear SVM. The nonlinear classifier can effectively expand the classification dimension and reduce the defects of softmax in nonlinear classification. For the training of the classifier, as shown in Figure 11 below, through the continuous iteration of the training samples, until the value of the optimization function reaches the optimal value, the classifier converges. The classifier can effectively expand the classification dimension. Taking SVM as an example, SVM projects the features into a high-dimensional space, and then non-linearly distinguishes the features. For linear classifiers such as Softmax and fully connected layers, only low-dimensional linear classification has a good effect. Using this scheme, the defect of Softmax in nonlinear classification can be reduced.

Therefore, the method of the embodiment of the present application solves the problem that it is difficult to manually determine the location of the local feature. At the same time, the extraction of global deep features is maintained. Through the combination of global and local features, the pain point of entity classification can be solved, and different types of entities in the image can be accurately identified, and different entities can be directional beautified when the user takes pictures, and entities such as food are also a large part of the camera function. Therefore, the application prospect is relatively large.

To sum up, the method for identifying entity categories in an image according to the embodiment of the present application extracts image features of the target image according to a preset algorithm, determines at least one target area containing the target entity in the target image according to the image features, and extracts at least one target area in the target image. The classification features of each target area, and fuse all the classification features to obtain local classification features, and then extract the global classification features of the target image, and fuse the local classification features and global classification features to obtain the target classification features, and finally, determine the target according to the target classification features. The entity class of the entity. Thus, the entity category of the target entity is determined according to the target classification feature.

In order to realize the above-mentioned embodiments, the present application also proposes a device for identifying entity categories in an image.

FIG. 12 is a schematic structural diagram of an apparatus for identifying entity categories in an image according to an embodiment of the present application.

As shown in FIG. 12 , the apparatus for identifying the entity category in the image includes: a first determination module 10 , a first fusion module 20 , a second fusion module 30 and a second determination module 40 .

Wherein, the first determination module 10 is configured to extract the image features of the target image according to a preset algorithm, and determine at least one target area including the target entity in the target image according to the image features;

The first fusion module 20 is used to extract the classification features of each target area in at least one target area, and fuse all the classification features to obtain local classification features;

The second fusion module 30 is used to extract the global classification feature of the target image, and obtain the target classification feature by fusing the local classification feature and the global classification feature;

The second determining module 40 is configured to determine the entity category of the target entity according to the target classification feature.

Further, in a possible implementation manner of the embodiment of the present application, the first determination module 10 is specifically used for:

According to the preset first convolution algorithm, obtain the first one-dimensional feature map of the target image;

Extracting the image features of the first one-dimensional feature map, and determining at least one candidate region where the target entity is located;

At least one candidate area is determined as at least one target area.

In this embodiment, the first determination module 10 is specifically configured to: extract a second one-dimensional feature map of each candidate region in the at least one candidate region according to a preset second convolution algorithm;

Extracting the image features of the first one-dimensional feature map, and converging at least one candidate region according to the image features of the first one-dimensional feature map;

The converged at least one candidate region is determined as at least one target region.

It should be noted that, the foregoing explanations on the embodiment of the method for recognizing entity types in images are also applicable to the device for recognizing entity types in images in this embodiment, which will not be repeated here.

To sum up, the device for recognizing the entity category in the image according to the embodiment of the present application extracts the image features of the target image according to the preset algorithm, determines at least one target area including the target entity in the target image according to the image features, and extracts at least one target area in the target image. The classification features of each target area, and fuse all the classification features to obtain local classification features, and then extract the global classification features of the target image, and fuse the local classification features and global classification features to obtain the target classification features, and finally, determine the target according to the target classification features. The entity class of the entity. Thus, the entity category of the target entity is determined according to the target classification feature.

In order to realize the above embodiments, the present application also proposes a terminal device.

FIG. 13 is a schematic structural diagram of a terminal device according to an embodiment of the present application. As shown in FIG. 13 , the terminal device 1300 may include: a memory 1302 , a processor 1304 , and a computer program 1306 stored in the memory 1303 and running on the processor 1304 . When the processor 1304 executes the computer program 406 , the above-mentioned application is implemented. The method for identifying entity categories in an image according to any one of the embodiments.

In order to implement the above-mentioned embodiments, the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the entity category in the image described in any one of the above-mentioned embodiments of the present application method of identification.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature delimited with "first", "second" may expressly or implicitly include at least one of that feature. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless expressly and specifically defined otherwise.

Any process or method description in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing custom logical functions or steps of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use with, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus, or apparatus) or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

It should be understood that various parts of this application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented by any one of the following techniques known in the art, or a combination thereof: discrete with logic gates for implementing logic functions on data signals Logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like. Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (10)

1. A method for identifying a category of an entity in an image, the method comprising:

extracting image features of a target image according to a preset algorithm, and determining at least one target area containing a target entity in the target image according to the image features;

extracting the classification characteristic of each target area in the at least one target area, and fusing all the classification characteristics to obtain a local classification characteristic;

extracting global classification features of the target image, and fusing the local classification features and the global classification features to obtain target classification features;

and determining the entity class of the target entity according to the target classification characteristic.

2. The method of claim 1, wherein the extracting image features of the target image according to a preset algorithm, and determining at least one target region containing a target entity in the target image according to the image features comprises:

acquiring a first one-dimensional feature map of the target image according to a preset first convolution algorithm;

extracting image features of the first one-dimensional feature map, and determining at least one candidate region where the target entity is located;

determining the at least one candidate region as the at least one target region.

3. The method of claim 2, wherein said determining the at least one candidate region as the at least one target region comprises:

extracting a second one-dimensional feature map of each candidate region in the at least one candidate region according to a preset second convolution algorithm;

extracting image features of the first one-dimensional feature map, and converging the at least one candidate region according to the image features of the first one-dimensional feature map;

determining the at least one candidate region after convergence as the at least one target region.

4. The method of claim 1, wherein said extracting classification features for each of said at least one target region comprises:

determining a confidence level of each target region containing the target entity;

extracting candidate classification features of each target region;

and determining corresponding classification features according to the candidate classification features and the confidence degrees of each target region.

5. The method of claim 1, wherein said extracting classification features for each of said at least one target region comprises:

determining the size of each target area;

sorting the at least one target area according to the size;

and sequentially inputting the at least one target area into a pre-trained fine classification model according to the sequencing result, and determining the classification characteristic of each target area.

6. An apparatus for identifying a category of an entity in an image, comprising:

the first determination module is used for extracting image characteristics of a target image according to a preset algorithm and determining at least one target area containing a target entity in the target image according to the image characteristics;

the first fusion module is used for extracting the classification characteristic of each target area in the at least one target area and fusing all the classification characteristics to obtain a local classification characteristic;

the second fusion module is used for extracting the global classification characteristic of the target image and fusing the local classification characteristic and the global classification characteristic to obtain a target classification characteristic;

and the second determining module is used for determining the entity class of the target entity according to the target classification characteristic.

7. The apparatus of claim 6, wherein the first determining module is specifically configured to:

acquiring a first one-dimensional feature map of the target image according to a preset first convolution algorithm;

extracting image features of the first one-dimensional feature map, and determining at least one candidate region where the target entity is located;

determining the at least one candidate region as the at least one target region.

8. The apparatus of claim 6, wherein the first determining module is specifically configured to:

extracting a second one-dimensional feature map of each candidate region in the at least one candidate region according to a preset second convolution algorithm;

extracting image features of the first one-dimensional feature map, and converging the at least one candidate region according to the image features of the first one-dimensional feature map;

determining the at least one candidate region after convergence as the at least one target region.

9. A terminal device, comprising: memory, processor and computer program stored on the memory and executable on the processor, the processor implementing a method of identifying a category of an entity in an image according to any one of claims 1 to 5 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out a method of identifying a category of an entity in an image according to any one of claims 1 to 5.

Images