WO2015070678A1

WO2015070678A1 - Image recognition method, and method and device for mining main body information about image

Info

Publication number: WO2015070678A1
Application number: PCT/CN2014/087954
Authority: WO
Inventors: 陶哲; 薛红霞; 白明; 韩玉刚
Original assignee: 北京奇虎科技有限公司; 奇智软件（北京）有限公司
Priority date: 2013-11-15
Filing date: 2014-09-30
Publication date: 2015-05-21

Abstract

An image recognition method and device. The method comprises: acquiring an image to be recognized, and searching for N other images similar thereto according to an image similarity; and acquiring main body information corresponding to other images, and, according to a similarity sequence, determining the weight of each of the other images, according to each piece of main body information and the weight of each corresponding image, respectively conducting a weight accumulation calculation on each piece of main body information, extracting main body information corresponding to the maximum accumulated value, and taking same as main body information about the image to be recognized. The solution of the present application can relatively accurately search out accurate description information about unknown images, and can then provide an accurate search result of unknown images for a user when mass image data exists in a network environment, thereby effectively improving the efficiency of processing of image data.

Description

Image recognition method, method and device for mining image body information

Technical field

The present application relates to the technical field of data processing, and in particular, to an image recognition method, a method and an apparatus for mining image body information.

Background technique

With the rapid development of the Internet and multimedia technologies, the image resources on the Internet are increasingly rich, and the image resources acquired from the network often contain a variety of information, such as background, time, place, subject, etc., and so much information. Under normal circumstances, it is not what the user really wants to pay attention to; for example, multiple images appear when browsing current news pages, and users may only pay attention to time and place for images in news; while users are browsing sports news pages. It may only focus on the characters and backgrounds in multiple images that appear.

At the same time, users can get a wide variety of images from a variety of sources, but not all images are accompanied by clear instructions or notes; for example, for users who are viewing sports news pages, in some cases, users The exact information of the image is not known; in addition, the user cannot obtain other images associated with the image based on the known image. At the same time, the image resources acquired by the user from the network often only have annotations or annotation information of the image, and the annotation or annotation information cannot accurately give the subject information of the image due to the massive information contained in the acquired image; for example, When a user browses a sports news webpage, the user can only guess the content to be expressed by the news headline and the article summary, and cannot accurately know the character information of the map.

Therefore, how to realize image recognition and image body information mining in a network environment, so as to accurately obtain an accurate description of the image or its associated subject information becomes very necessary and urgent.

Summary of the invention

In view of the above problems, the present application has been made in order to provide an image recognition method, a method for mining image body information, and a corresponding device that overcome the above problems or at least partially solve or alleviate the above problems.

According to an aspect of the present application, a method for image recognition is provided, including:

Obtaining an image to be identified, searching for N other images similar to the image similarity degree; acquiring body information corresponding to the other images; and determining a weight value of each other image according to the similarity order, according to the weight information of each body information and the corresponding image, The weight information is separately calculated for each subject information, and the subject information corresponding to the largest accumulated value is extracted as the subject information of the image to be identified.

According to another aspect of the present application, a method for mining image body information is provided, including:

Acquiring an image and its annotation information; acquiring a support information list of the image annotation information by using the training data; and extracting the body information of the image from the support information list.

According to still another aspect of the present application, an apparatus for image recognition is provided, including:

a search module, configured to obtain an image to be recognized, and find N other images similar to the image similarity; the sorting module is configured to acquire body information corresponding to other images found by the searching module, and determine each order according to similarity ranking The weighting value of the other images; the calculating module is configured to perform weight accumulation calculation on each body information according to the weight information of each body information and the corresponding image; and the identifying module is configured to extract the body information corresponding to the largest accumulated value, as the waiting Identify the subject information of the image.

According to still another aspect of the present application, an apparatus for mining image body information includes:

Obtaining a module, configured to acquire an image and its annotation information; a generation module, configured to acquire a support information list of the image annotation information by using training data; and an extraction module configured to extract body information of the image from the support information list .

According to still another aspect of the present application, there is provided a computer program comprising computer readable code, when said computer readable code is run on a computing device, causing said computing device to perform according to claims 1-15 Any of the methods described.

According to still another aspect of the present application, there is provided a computer readable medium storing the computer program of claim 31.

The beneficial effects of the present application are: obtaining the image to be recognized and searching for N other images similar to each other according to the image similarity, and then acquiring the body information corresponding to the other images and determining the weight of each other image according to the similarity order, according to Each subject information and the weight of the corresponding image are respectively subjected to weight accumulation calculation for each subject information, and the subject information corresponding to the largest accumulated value is extracted as the subject information of the image to be identified; thereby, an accurate description of the unknown image can be relatively accurately searched. Information, and thus in the presence of massive image data in a network environment The user can provide the accurate result of the unknown image search, and effectively improve the efficiency of the image data processing; in addition, by acquiring the image and its annotation information, and using the training data to obtain the support information list of the image annotation information, and then from the The main body information of the image is extracted from the support information list; thereby, the main body information of the image can be excavated relatively accurately, and the unnecessary interference description in the image annotation information or the annotation information is excluded, thereby improving the accuracy of the data search.

The above description is only an overview of the technical solutions of the present application, and the technical means of the present application can be more clearly understood, and the above and other objects, features and advantages of the present application can be more clearly understood. The following is a specific embodiment of the present application.

DRAWINGS

Various other advantages and benefits will become apparent to those skilled in the art from a The drawings are only for the purpose of illustrating the preferred embodiments and are not intended to be limiting. Throughout the drawings, the same reference numerals are used to refer to the same parts. In the drawing:

1 is a flow chart showing the steps of an embodiment of a method for image recognition according to an embodiment of the present application;

2 is a flow chart showing the steps of an embodiment of a method for mining image body information according to an embodiment of the present application;

3 is a block diagram showing the structure of an apparatus for image recognition according to an embodiment of the present application;

4 is a block diagram showing the structure of an embodiment of a sorting module according to an embodiment of the present application;

FIG. 5 is a schematic block diagram showing an embodiment of an apparatus for mining image body information according to an embodiment of the present application; FIG.

Figure 6 shows schematically a block diagram of a computing device for performing the method according to the present application;

Fig. 7 schematically shows a storage module for holding or carrying program code implementing the method according to the present application.

Specific embodiment

The present application is further described below in conjunction with the drawings and specific embodiments.

1 is a flow chart showing the steps of Embodiment 1 of an image recognition method according to an embodiment of the present application, which may specifically include the following steps:

Step 110: Acquire an image to be identified, and find N other images similar to the image according to the image similarity;

Specifically, after receiving the image recognition request, the remote computing device extracts the image to be identified from the image recognition request; constructs an inverted index by using the similar feature of the image, and then performs similar search on the image to be identified to obtain a similar image. N other images. Of course, those skilled in the art can easily understand that other methods can be used to find other N images similar to each other according to the image similarity. For example, the face recognition technology can be used to extract the character frame of the image to be recognized, and then the character is used. The framework finds N other images similar to each other; in addition, we can also aggregate the existing image similarity to form a cluster of aggregated images, and obtain a similar N according to the aggregated image cluster to which the image to be identified belongs. Other images. Image similarity aggregation techniques such as sift/surf, phash, haar, or some CSD-7 CSD, SCD, CLD, DCD, HTD, EHD.

Step 120: Acquire main body information corresponding to other images, and determine weight values of each other image according to similarity order;

In this embodiment, after the other images are found, the method of the following embodiment 2 can be used to obtain the body information corresponding to the other images. Of course, those skilled in the art can easily understand that there are other methods for acquiring the image body information. This will not be repeated here;

The similarity relationship between the other N chapter images found and the image to be recognized is

Where x is the reference value of the image pixel to be identified, μ is the reference value of other image pixels, and when x=μ, f(x)=1;

The above formula is only an example of an embodiment of the technical solution of the present invention, and various modifications are made to the formula by those skilled in the art, and the essence thereof is also within the protection scope of the present invention.

Let the similarity of each image to the image to be recognized be σμ in the similarity of the similarity of the image to be recognized in all the images, then the weight of the image is:

Step 130: Perform weight calculation on each subject information according to the weight information of each subject information and the corresponding image;

Assuming that the number of images in which the subject information is Name is M in N other images, and the weight of the image is determined to be Weight _i , the weights of the image whose subject information is Name are cumulatively processed as:

Step 140: Extract body information corresponding to the maximum accumulated value as the body information of the image to be identified.

After obtaining the weight accumulation result of all the N other images, the object is sorted according to the weight value, and the body information corresponding to the image with the largest weight accumulation result is extracted therefrom, and the body information is used as the body information of the image to be identified.

The embodiment of the present application obtains an image to be identified and searches for N other images similar to each other according to the image similarity, and then acquires the body information corresponding to the other images, and determines the weight of each other image according to the similarity order, according to each subject information. And the weight of the corresponding image, respectively performing weight calculation on each subject information, extracting the subject information corresponding to the largest accumulated value, as the subject information of the image to be identified; thereby accurately searching for accurate description information of the unknown image, and further In the network environment where massive image data exists, it can provide users with accurate results of unknown image search, and effectively improve the efficiency of image data processing.

Referring to FIG. 2, a flow chart of steps of an embodiment of a method for acquiring body information corresponding to another image according to another embodiment of the present application is shown, which may specifically include the following steps:

Step 210: Acquire an image and its annotation information;

As is well known, the image acquired by the user often contains a variety of information, for example, a picture may contain N names (N>=0), and only one of the names is required by the user, and the name is often Included in the description of the picture, the description letter The information is the annotation information of the picture; in addition, some images are actually inconsistent with the annotation information, such as the picture "Some movie festival is amazing," where "Some movie festival is amazing" is the picture The information is marked, but the image may show a lot of images of other people in the film festival. In this case, the user cannot accurately know the subject information of the image from the annotation information. Therefore, when the remote computing device receives such an image of the external input and its annotation information, the annotation information of the image is directly extracted for subsequent processing. If the remote computing device only receives the externally input image but does not receive the annotation information of the image, it searches for the annotation information of the image according to the URL of the image, the image source, the text surrounding the image, and the like, and stores the image. For subsequent processing;

Of course, it is easy for those skilled in the art to know that the image information of the image can be obtained by other means after the image is received.

Step 220: Obtain a support information list of the image annotation information by using training data.

Since the information included in the image tagging information is large, in order to avoid the error of extracting the body information of the image, the embodiment proposes to obtain a specific image body by acquiring the support information list of the image tagging information. Information, which can effectively improve the accuracy of image description;

Specifically, in this embodiment, the following information is used to obtain a list of support information for acquiring image annotation information by using training data, but is not limited thereto:

S221: Acquire intermediate data of the image annotation information;

The annotation information of the image is often a collection of multiple information, but usually the collection of the multiple information contains intermediate data; in practical applications, the intermediate data is usually the subject of the image annotation information or the annotation, and the part of speech is often For nouns, such as names of people, places, etc.; for example, if an annotated information is an image of "someone's girlfriend", the intermediate data is "some"; at the same time, because the selection of the intermediate data of the image is usually The words around the word are determined, for example, in the image tagging information "someone's girlfriend", "someone" should be selected separately, which is the intermediate data, and the "girlfriend" should not be selected, which is As a supporting word; specifically, in the annotation information of an image, the entropy of the left and right words of the intermediate data is much smaller than the entropy of the left and right words of the supporting words, so by comparing the entropy of the words of any word in the annotation information, Determine if the word is intermediate data.

S222: Extract training data related to the intermediate data from an image database;

It should be noted that, in order to facilitate the search, an image database is preset in the embodiment, and a plurality of intermediate data and supporting words are stored in the image database, and all supporting words are collectively referred to as training. Data; and the plurality of intermediate data and the supporting words have a corresponding matching relationship in the image database, that is, an intermediate data corresponds to multiple supporting words, and the same data may exist in the supporting words corresponding to the plurality of intermediate data; of course, the field It is easy for a person skilled in the art to understand that the image database can be extracted from the existing network database without a preset image database. Of course, the matching relationship of the data in the preset image database can also be diverse. Let me repeat.

Specifically, after acquiring the intermediate data in the image annotation information, searching the preset image database with the intermediate data as a target, and if the intermediate data exists in the image database, extracting the middle by using the matching relationship The relevant supporting word corresponding to the data; otherwise, the operation is ended and the indication of the image without the subject information is externally fed back.

S223: Calculate a correlation score of the training data and the intermediate data;

It should be noted that since the correlation between the intermediate data and its training data is a positively distributed relationship, the correlation score between the training data and the intermediate data can be calculated by using this rule.

S224: Generate a support information list of the image annotation information by using the correlation score.

Specifically, since the intermediate data often corresponds to multiple supporting words, when the correlation scores of the related supporting words and the intermediate data are calculated, the correspondence between the training data and the intermediate data relevance scores can be matched, and then the vocabulary is supported. After obtaining the correspondence between the plurality of sets of training data and the intermediate data relevance scores, the list consisting of the plurality of sets of supporting vocabularies is the support information list of the image annotation information.

Step 230: Extract body information of the image from the support information list.

It is worth noting that the correlation between the intermediate data and the supporting words composing the training data is a normal distribution relationship. This embodiment also proposes a method for calculating the correlation score between the training data and the intermediate data, but it is not Limited to this, the specific method includes:

S2231: Calculate the correlation weight of the training data and the intermediate data and sum E1;

Specifically, the normal distribution relationship between the intermediate data and the supporting words is

Where x is the intermediate data and μ is the supporting word;

The above formula is only an example of an embodiment of the technical solution of the present invention, and various modifications are made to the formula by those skilled in the art, and the essence thereof is also in the protection scope of the present invention. Inside.

Therefore, by using the correlation between μ and x as the weight, the score of each support word can be obtained, and then the scores of each support word can be added to obtain the total score of the training data, as follows:

When x=μ, f(x)=1, the correlation weight of the support words with the intermediate data x distance step iσμ is:

Where σμ is the step size of the unit support word and the intermediate data distance;

It should be noted that the above formula is only an example of implementing one embodiment of the technical solution of the present invention, and various modifications are made to the formula by those skilled in the art, and the essence thereof is also within the protection scope of the present invention.

It can also be seen that the farther the support word from the intermediate data is, the worse the correlation is, and the smaller the weight is;

Then, the scores of the N support words are accumulated to obtain the correlation weight E1 of all the training data and the intermediate data:

S2232: accumulating the correlation weights of all the training data and the intermediate data and summing E2;

If the intermediate data has multiple sets of training data M, the sum of the correlation weights of all the training data and the intermediate data is E2:

S2233: Determine a first correlation score of the training data and the intermediate data by calculating a ratio of the E2 to the E1;

Specifically, the first correlation score of each set of training data and the intermediate data is E3:

In addition, this embodiment also proposes to extract the body information of the image from the support information list by the following manner, but is not limited thereto;

S231: Acquire all intermediate data of the image annotation information and related training data;

S232: Calculate a score of each intermediate data by counting scores of the same training data in the support information list;

Specifically, it is assumed that the total number of related training data of each intermediate data Name in all the intermediate data is P, and the score of the supporting word Word _i in the training information word table in the supporting information word table is Score _i , and the weight thereof is Weight _i . Then the score of the intermediate data Name is:

S233: determining a size of each intermediate data and a preset threshold. When the score of an intermediate data is not less than the preset threshold, determining that the intermediate data is a subject name of the image; otherwise, determining There is no subject name for this image.

In addition, the embodiment further provides another method for acquiring subject information corresponding to other images, and the method further includes the following steps on the basis of the foregoing method:

Step 240: Perform denoising processing of the training data after determining a first correlation score of the training data and the intermediate data;

It should be noted that in the relevant training data obtained, there are usually some support words that have no practical meaning but are related to the intermediate data in the image database. These support words are often just ordinary data, which are used as the subject name. The probability is very low; for example, vocabulary such as person name and place name can often be used as supporting words, but words such as "" and "and" are supporting words of adverbs, and the probability of being the subject name is very low; in this embodiment, this is the case. section Support words are defined as background noise, which affects the accuracy of supporting vocabularies.

Specifically, the denoising process in this embodiment can be implemented in the following manner, including:

S241: Calculate the correlation weights of the other training data that are synchronous with any training data and all the training data, and perform the summation processing and summing to determine the noise weight B1 of the any training data;

Specifically, the calculation of the background noise value in this embodiment is similar to the calculation method of the weight of the support word, and of course, there may be other ways of calculating; using σμ as the unit step size, the support word with the distance of any supporting word step is iσμ. The weights are as follows:

The weight of the same supporting word that is synchronized with any supporting word is accumulated, and the background noise weight B1=BackNoise _{word of} the arbitrary supporting word is obtained;

S242: Accumulating the noise of all the training data and summing to determine the total noise weight of all the training data B2=Total _BackNoise ;

S243: Determine a second relevance score of the training data and the intermediate data by acquiring a difference between the first correlation score and a noise value of the training data; wherein, the F1 and the F2 The ratio is the noise value of the training data.

Specifically, after calculating the background noise weight F1 of any support word and the total noise weight F2 of all training data, the background noise value of each supported word is obtained:

The denoising process in this embodiment can be implemented by making the basis score of each supporting word and the background noise value of the supporting word, that is, the score of the supporting word after denoising is:

Score=Score _{sup port} -Score _BackNoise

Then record the scores of all supporting words and finally generate a supporting vocabulary.

Of course, the above-mentioned special information and its judgment manner are only examples. In the implementation of the embodiment of the present application, other special information and its judgment manner may be set according to actual conditions, which is not limited by the embodiment of the present application. In addition, in addition to the above-mentioned special information and its judgment manner, the person skilled in the art can also adopt other special information and its judgment manner according to actual needs, and the embodiment of the present application does not limit this.

The embodiment of the present application obtains the image and the annotation information thereof, and obtains the support information list of the image annotation information by using the training data, and then extracts the body information of the image from the support information list; The subject information of the image excludes unnecessary interference descriptions in the image annotation information or annotation information, and improves the accuracy of the data search.

For the method embodiments, for the sake of brevity, they are all described as a series of action combinations, but those skilled in the art should understand that the embodiments of the present application are not limited by the described action sequence, because the embodiment according to the present application Some steps can be performed in other orders or at the same time. In the following, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required in the embodiments of the present application.

Referring to FIG. 3, the present application further discloses an apparatus for image recognition, including the following module: a search module 310, configured to acquire an image to be recognized, and find N other images similar to the image according to the image similarity; the sorting module 320, set Obtaining the body information corresponding to the other images found by the searching module, and determining the weights of each of the other images according to the similarity ranking; the calculating module 330 is configured to respectively perform the objects according to the weights of the main body information and the corresponding images. The information is subjected to weight accumulation calculation; and the identification module 340 is configured to extract the body information corresponding to the maximum accumulated value as the body information of the image to be identified.

The searching module 310 in this embodiment includes (not shown in the figure): a receiving module that receives an image recognition request, and an extracting module that extracts an image to be recognized from an image recognition request received by the receiving module. .

In addition, the lookup module 310 can include (not shown):

An indexing module, configured to build an inverted index by using similar features of the image;

The comparison module performs the similar retrieval on the image to be identified, and acquires N other images similar thereto.

It is to be noted that, in the apparatus for image recognition in this embodiment, as shown in FIG. 4, the sorting module 320 may specifically include the following modules: an acquiring module 410 configured to acquire an image and its annotation information; and a generating module 420 configured to The support information list of the image annotation information is acquired by using the training data; and the extraction module 430 is configured to extract the body information of the image from the support information list.

The generating module 420 includes: (not shown in the figure): a first processing module configured to acquire intermediate data of the image annotation information acquired by the acquiring module; and a second processing module configured to extract from the image database a training data associated with the intermediate data; a third processing module configured to calculate a first correlation score of the training data and the intermediate data; and a fourth processing module configured to utilize the first correlation score The value generates a list of support information for the image annotation information.

It should be noted that, in the embodiment, the third processing module may also include (not shown): the first calculator is configured to calculate and calculate the correlation weights of the training data and the intermediate data. E1; a second calculator configured to accumulate correlation weights of all training data and the intermediate data and sum E2; and a third calculator configured to determine by calculating a ratio of the E2 to the E1 The first correlation score of the training data and the intermediate data.

In addition, the apparatus for mining image body information of the embodiment may further include (not shown): a denoising module, configured to perform the training data after the third processing module determines the first relevance score Denoising processing.

Specifically, the embodiment provides that the denoising module may further include (not shown): a fourth calculator configured to calculate correlation between other training data that is synchronous with any training data and all training data. Weight, the correlation weight is cumulatively processed and summed to determine the noise weight F1 of any training data; the fifth calculator is set to accumulate the noise of all training data and summed to determine all a total noise weight F2 of the training data; a sixth calculator configured to determine a second correlation between the training data and the intermediate data by obtaining a difference between the first correlation score and a noise value of the training data a sex score; wherein the ratio of the F1 to the F2 is a noise value of the training data.

In addition, the extraction module of the embodiment may further include (not shown): a fifth processing module, configured to acquire all intermediate data of the image annotation information and the phase thereof And a sixth processing module, configured to calculate a score of each intermediate data by counting scores of the same training data in the support information list; and a seventh processing module configured to determine a score of each of the intermediate data And the size of the preset threshold, when the score of an intermediate data is not less than the preset threshold, determining that the intermediate data is the body information of the image.

Referring to FIG. 5, a structural block diagram of an apparatus for mining image body information according to an embodiment of the present application is shown. Specifically, the method may include the following modules: an acquiring module 510 configured to acquire an image and labeling information thereof; and a generating module 520. And a set of support information for acquiring the image annotation information by using the training data; and an extraction module 530, configured to extract the body information of the image from the support information list.

The generating module 520 includes: (not shown in the figure): the first processing module is configured to acquire intermediate data of the image annotation information acquired by the acquiring module; and the second processing module is configured to extract from the image database a training data associated with the intermediate data; a third processing module configured to calculate a first correlation score of the training data and the intermediate data; and a fourth processing module configured to utilize the first correlation score The value generates a list of support information for the image annotation information.

In addition, in the apparatus for mining image body information of the embodiment, the extraction module may further include (not shown): a fifth processing module configured to acquire all intermediate data of the image annotation information and Corresponding training data; a sixth processing module, configured to calculate a score of each intermediate data by counting scores of the same training data in the support information list; and a seventh processing module configured to determine each of the intermediate data The score is equal to the size of the preset threshold. When the score of an intermediate data is not less than the preset threshold, it is determined that the intermediate data is the body information of the image.

The various component embodiments of the present application can be implemented in hardware, or in a software module running on one or more processors, or in a combination thereof. It will be understood by those skilled in the art that a 徼 processor or a digital signal processor (DSP) may be used in practice to implement some or all of the components of the image recognition device or the device for exchanging image body information according to embodiments of the present application. Some or all of the features. The application can also be implemented as a device or device program (eg, a computer program and a computer program product) configured to perform some or all of the methods described herein. Such a program implementing the present application may be stored on a computer readable medium or may be in the form of one or more signals. Such signals may be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

For example, FIG. 6 illustrates a computing device, such as a client terminal device or server, that may implement an image recognition method or a method of mining image body information in accordance with the present application. The computing device conventionally includes a processor 610 and a computer program product or computer readable medium in the form of a memory 620. The memory 620 may be an electronic memory such as a flash memory, an EEPROM (Electrically Erasable Programmable Read Only Memory), an EPROM, a hard disk, or a ROM. Memory 620 has a memory space 630 of program code 631 that is configured to perform any of the method steps described above. For example, the storage space 630 for program code may include respective program codes 631 that are respectively set to implement various steps in the above method. The program code can be read from or written to one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. Such computer program products are typically portable or fixed storage modules as described with reference to FIG. The storage module can have storage segments, storage spaces, and the like that are similarly arranged to memory 620 in the computing device of FIG. The program code can be compressed, for example, in an appropriate form. Typically, the storage module includes computer readable code 631', i.e., a generation that can be read by a processor such as 610. Codes that, when executed by a computing device, cause the computing device to perform various steps in the methods described above.

"an embodiment," or "an embodiment," or "one or more embodiments" as used herein means that the particular features, structures, or characteristics described in connection with the embodiments are included in at least one embodiment of the present application. In addition, it is noted that the phrase "in one embodiment" is not necessarily referring to the same embodiment.

In the description provided herein, numerous specific details are set forth. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of the description.

It should be noted that the above-described embodiments are illustrative of the present application and are not intended to limit the scope of the application, and those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as a limitation. The word "comprising" does not exclude the presence of the elements or steps that are not recited in the claims. The word "a" or "an" The application can be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means can be embodied by the same hardware item. The use of the words first, second, and third does not indicate any order. These words can be interpreted as names.

In addition, it should be noted that the language used in the specification has been selected for the purpose of readability and teaching, and is not intended to be interpreted or limited. Therefore, many modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The disclosure of the present application is intended to be illustrative, and not restrictive, and the scope of the application is defined by the appended claims.

Claims

A method of image recognition, comprising:

Obtaining an image to be identified, and finding N other images similar to the image according to image similarity;

Obtaining body information corresponding to other images and determining weights of each other image according to similarity ordering,

According to the weight information of each subject information and the corresponding image, weight calculation is performed on each subject information,

The body information corresponding to the largest accumulated value is extracted as the body information of the image to be identified.
The method of claim 1, wherein the obtaining the image to be identified comprises:

Receiving an image recognition request;

An image to be identified is extracted from the image recognition request.
The method according to claim 1 or 2, wherein the finding N other images similar to the image similarity includes:

The inverted index is built by the similar feature of the picture, and then the image to be recognized is similarly searched to obtain N other images similar thereto.
The method according to claim 1, wherein the subject information corresponding to the other images is obtained by:

Obtain images and their annotation information;

Obtaining a support information list of the image annotation information by using training data;

The body information of the image is extracted from the list of support information.
The method of claim 4, wherein the obtaining the support information list of the image annotation information by using the training data comprises:

Obtaining intermediate data of the image annotation information;

Extracting training data related to the intermediate data from an image database;

Calculating a first relevance score of the training data and the intermediate data;

Generating a support information list of the image annotation information by using the first correlation score.
The method of claim 5, wherein the calculating the relevance score of the training data and the intermediate data comprises:

Calculating a correlation weight of the training data and the intermediate data and summing E1;

All the training data and the intermediate data correlation weights are cumulatively processed and summed E2;

A first correlation score of the training data and the intermediate data is determined by calculating a ratio of the E2 to the E1.
The method of claim 6 further comprising:

Denoising processing of the training data is performed after determining a first correlation score of the training data and the intermediate data.
The method of claim 7, wherein the performing the denoising processing of the training data comprises:

Calculating a correlation weight of the other training data that is synchronous with any training data and all the training data, accumulating the correlation weights and summing the noise weight F1 of the any training data;

The noise of all the training data is accumulated and summed to determine the total noise weight F2 of all the training data;

Determining, by acquiring a difference between the first correlation score and a noise value of the training data, a second relevance score of the training data and the intermediate data; wherein, a ratio of the F1 to the F2 is The noise value of the training data.
The method of claim 5, the extracting the body information of the image from the support information list comprises:

Obtaining all intermediate data of the image annotation information and related training data;

Calculating the score of each intermediate data by counting the scores of the same training data in the support information list;

Determining the score of each of the intermediate data and the size of the preset threshold. When the score of an intermediate data is not less than the preset threshold, determining that the intermediate data is the body information of the image.
A method for mining image body information, comprising:

Obtain images and their annotation information;

Obtaining a support information list of the image annotation information by using training data;

The body information of the image is extracted from the list of support information.
The method of claim 10, wherein the obtaining the support information list of the image annotation information by using the training data comprises:

Obtaining intermediate data of the image annotation information;

Extracting training data related to the intermediate data from an image database;

Calculating a first relevance score of the training data and the intermediate data;

Generating a support information list of the image annotation information by using the first correlation score.
The method of claim 11 wherein said calculating training data and said intermediate number The first relevance scores are:

Calculating a correlation weight of the training data and the intermediate data and summing E1;

All the training data and the intermediate data correlation weights are cumulatively processed and summed E2;

A first correlation score of the training data and the intermediate data is determined by calculating a ratio of the E2 to the E1.
The method of claim 11 further comprising:

Denoising processing of the training data is performed after determining a first correlation score of the training data and the intermediate data.
The method of claim 13, wherein the performing the denoising processing of the training data comprises:

Calculating a correlation weight of the other training data that is synchronous with any training data and all the training data, accumulating the correlation weights and summing the noise weight F1 of the any training data;

The noise of all the training data is accumulated and summed to determine the total noise weight F2 of all the training data;

Determining, by acquiring a difference between the first correlation score and a noise value of the training data, a second relevance score of the training data and the intermediate data; wherein, a ratio of the F1 to the F2 is The noise value of the training data.
The method according to claim 11, wherein the extracting the body information of the image from the support information list comprises:

Obtaining all intermediate data of the image annotation information and related training data;

Calculating the score of each intermediate data by counting the scores of the same training data in the support information list;

Determining the score of each of the intermediate data and the size of the preset threshold. When the score of an intermediate data is not less than the preset threshold, determining that the intermediate data is the body information of the image.
An image recognition device comprising:

a search module, configured to acquire an image to be recognized, and find N other images similar to the image according to the image similarity;

a sorting module, configured to obtain body information corresponding to other images found by the searching module, and determine weights of each other image according to similarity ordering,

The calculation module is set to respectively according to the weight information of each subject information and the corresponding image The body information is calculated by adding weights,

The identification module is configured to extract body information corresponding to the maximum accumulated value as the body information of the image to be identified.
The apparatus of claim 16, the lookup module comprising:

a receiving module that receives an image recognition request, and

An extraction module that extracts an image to be recognized from an image recognition request received by the receiving module.
The apparatus of claim 16, the lookup module comprising:

An indexing module, configured to build an inverted index by using similar features of the image;

The comparison module performs the similar retrieval on the image to be identified, and acquires N other images similar thereto.
The apparatus of claim 16, the ordering module comprising:

Obtain a module, set to obtain an image and its annotation information;

Generating a module, configured to obtain a support information list of the image annotation information by using training data;

An extraction module is configured to extract body information of the image from the list of support information.
The apparatus of claim 19, the generating module comprising:

a first processing module, configured to acquire intermediate data of the image annotation information acquired by the acquiring module;

a second processing module configured to extract training data related to the intermediate data from an image database;

a third processing module, configured to calculate a first correlation score of the training data and the intermediate data;

The fourth processing module is configured to generate a support information list of the image annotation information by using the first correlation score.
The apparatus of claim 20, the third processing module comprising:

a first calculator, configured to calculate a correlation weight of the training data and the intermediate data and sum E1;

a second calculator, configured to accumulate the correlation weights of all the training data and the intermediate data and sum E2;

a third calculator configured to determine the training by calculating a ratio of the E2 to the E1 A first correlation score of the training data and the intermediate data.
The apparatus of claim 20, further comprising:

And a denoising module, configured to perform denoising processing of the training data after the third processing module determines the first correlation score.
The apparatus of claim 22, the denoising module comprising:

a fourth calculator configured to calculate a correlation weight of the other training data and all the training data that are synchronous with any of the training data, perform the cumulative processing of the correlation weights, and determine the training data after the summation Noise weight F1;

a fifth calculator, configured to accumulate the noise of all the training data and summed to determine the total noise weight F2 of all the training data;

a sixth calculator configured to determine a second relevance score of the training data and the intermediate data by obtaining a difference between the first correlation score and a noise value of the training data; wherein the F1 The ratio to the F2 is the noise value of the training data.
The apparatus of claim 19, the extraction module comprising:

a fifth processing module, configured to acquire all intermediate data of the image annotation information and related training data;

a sixth processing module, configured to calculate a score of each intermediate data by counting scores of the same training data in the support information list;

The seventh processing module is configured to determine a score of each of the intermediate data and a size of a preset threshold. When a score of the intermediate data is not less than the preset threshold, determining that the intermediate data is the body information of the image.
An apparatus for mining image body information, comprising:

Obtain a module, set to obtain an image and its annotation information;

Generating a module, configured to obtain a support information list of the image annotation information by using training data;

An extraction module is configured to extract body information of the image from the list of support information.
The device of claim 25, wherein the generating module comprises:

a first processing module, configured to acquire intermediate data of the image annotation information acquired by the acquiring module;

a second processing module configured to extract training data related to the intermediate data from an image database;

a third processing module, configured to calculate a first correlation score of the training data and the intermediate data;

The fourth processing module is configured to generate a support information list of the image annotation information by using the first correlation score.
The device of claim 26, wherein the third processing module comprises:

a first calculator, configured to calculate a correlation weight of the training data and the intermediate data and sum E1;

a second calculator, configured to accumulate the correlation weights of all the training data and the intermediate data and sum E2;

And a third calculator configured to determine a first relevance score of the training data and the intermediate data by calculating a ratio of the E2 to the E1.
The device of claim 26, further comprising:

And a denoising module, configured to perform denoising processing of the training data after the third processing module determines the first correlation score.
The apparatus of claim 28, the denoising module comprising:

a fourth calculator configured to calculate a correlation weight of the other training data and all the training data that are synchronous with any of the training data, perform the cumulative processing of the correlation weights, and determine the training data after the summation Noise weight F1;

a fifth calculator, configured to accumulate the noise of all the training data and summed to determine the total noise weight F2 of all the training data;

a sixth calculator configured to determine a second relevance score of the training data and the intermediate data by obtaining a difference between the first correlation score and a noise value of the training data; wherein the F1 The ratio to the F2 is the noise value of the training data.
The apparatus of claim 25, the extraction module comprising:

a fifth processing module, configured to acquire all intermediate data of the image annotation information and related training data;

a sixth processing module, configured to calculate a score of each intermediate data by counting scores of the same training data in the support information list;

The seventh processing module is configured to determine a score of each of the intermediate data and a size of a preset threshold. When a score of the intermediate data is not less than the preset threshold, determining that the intermediate data is the body information of the image.
A computer program comprising computer readable code that, when executed on a computing device, causes the computing device to perform the method of any of claims 1-15.
A computer readable medium storing the computer program of claim 31.