CN105824886A - Rapid food recognition method based on Markov random field - Google Patents

Abstract

The invention discloses a rapid food recognition method based on a Markov random field, and mainly solves the problem that food is difficult to recognize due to the conditions that the food lacks of feature points and the shapes of the food are not fixed in food image recognition. The realization process of the rapid food recognition method comprises the following steps: (1) establishing a retrieval database, extracting feature descriptors in retrieval images, and constructing index files; (2) extracting feature descriptors of queried images; (3) calculating out the likelihood score of the queried images in food label types of the retrieval base according to the feature descriptors; (4) according to the likelihood score of the queried images in the label type food in the retrieval database and according to the condition probability between labels in menus of queried people, constructing a Markov energy formula and a smallest energy formula so as to obtain the food label type of the queried images. According to the rapid food recognition method disclosed by the invention, rapid and accurate recognition and classification can be performed on the food images, the food images can be easily expanded to a larger retrieval database and a queried image set, and the rapid food recognition method can be used for cultivating a healthy diet manner.

Description

Fast food based on markov random file recognition methods

Technical field

The invention belongs to Image Information Processing field, a kind of food recognition methods, can be used for solving food image analysis and identifying food qualification category problem.

Background technology

Along with people's concern to healthy diet, for those people wanting to improve oneself dietary habit, record diet information by a simple method and be very important.Up to the present, 24 hours whole day diet records still use the method for common record " diet history " to assess individual dietary habit.But, the method for self-report is the most inaccurate, and particularly in Overweight people, some people generally conceals the energy intake of oneself.If the nutritional information of diet can be crossed and automatically be detected by food image, user can be freed from manual record information.

In the recent period, in " life daily record (life-log) " research field, people are more and more denseer to the interest obtaining, processing people's daily life information.People record their life by various modes: upload pictures, upload gps data etc., and even, some people can share oneself " life daily record " on the internet.Up to the present, major part focuses in universal multimedia based on " life daily record " data process about the research of " life daily record ".In these are studied, when people are always at table, use photographing unit shooting food photo.Along with food photo ever-increasing in the Internet, they are devoted to collect, process the data that all available arrives, and set up the index of " life daily record ", summary, and can retrieve " life daily record " data of magnanimity.But the food in identification image is remarkable, because " life daily record " the most numerous and the most jumbled, redundancy.

Food identifies it is the succedaneum of a kind of manual record method automatically, if there being the method according to image automatic identification food, people just can easily differentiate the diet trend of oneself, and the photo shot recently without browsing pictures library lookup.But research in terms of food identification is little.In current research method, some existing recognizers achieve good effect to the identification of general objectives object, and some of which employs based on local feature, such as SIFT descriptor, some employs based on global characteristics, such as color histogram and GIST characteristic point；Yang et al. will utilize two kinds of baseline algorithm, and the PFID data base of color histogram and bagofSIFT feature is incorporated into food identification field.Felzenszwalb et al. proposes the technology using trigonometric ratio polygon to represent indefinite shape.Leordeanu et al. proposes and a kind of utilizes the paired simple descriptor of interaction to identify target class method for distinguishing.First, these methods need to detect significant characteristic point, such as edge, profile, key point and terrestrial reference to Arecentwork [8] learnsameanshapeoftheobjectclassbasedonthethinplatesplin eparameterization..But, owing to food image lacks above-mentioned significant characteristic point, secondly as the shape of real food is the most unformed, so being difficult to remove to judge the similarity of food shapes in aforementioned manners, it is impossible to accurately food is classified.

Summary of the invention

Present invention aims to above-mentioned existing technical problem, the recognition methods of a kind of fast food based on markov random file is proposed, with by query image the Likelihood Score of food label apoplexy due to endogenous wind and the prior probability of label classification ask query image at the Markov energy equation of food label apoplexy due to endogenous wind, and by optimize energy equation determine can be minimum food label classification.

To achieve these goals, technical scheme includes the following:

(1) searching database is set up according to the food image of different food species, the retrieval image D from searching database_dMiddle extraction D_dN kind feature descriptorAnd store it in index file, D_dRepresent the d retrieval image, d=1,2 ..., N_d, N_dRepresent and retrieve total number of images, k=1,2 ... N, N represent the sum of the feature descriptor of use；

(2) set i-th and be queried image s_i, and extract s_iKth kind feature descriptorI=1,2 ..., N_s, N_sRepresent and be queried total number of images；

(3) according to retrieval image and the feature descriptor being queried image, calculate i-th and be queried image s_iIn jth label class c_jLikelihood Score P (s in food_i|c_j), j=1,2 ..., N_c, N_cThe sum of the food label classification in expression searching database；

(4) structure is queried food image set SP={s_i| i=1,2 ..., N_sFood label class set c={c in searching database_j| j=1,2 ..., N_cMarkov MRF energy equation J (c) formula in },, and by minimizing Markov MRF energy J (c), show that i-th is queried image s_iFood label class c identified_v, v=1,2 ... N_c:

(4.1) m-th label class c in the person's of being queried menu is calculated_mFood is in the n-th label class c_nProbability of occurrence P (c under food Conditions_m|c_n) n, m=1,2 ..., N_p, n ≠ m, N_pRepresent the sum of food label classification in the person's of being queried menu；

(4.2) the n-th label class c in the person's of being queried menu is sought_nFood is in m-th label class c_mProbability of occurrence P (c under food Conditions_n|c_m)；

(4.3) conditional probability { P (c between any two food label class in the double counting person of being queried menu_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p}；

(4.4) it is queried the image Likelihood Score { P (s at food label apoplexy due to endogenous wind according to be queried in image collection SP_i|c_j) | i=1,2 ..., N_s, j=1,2 ..., N_cAnd the person's of being queried menu in conditional probability { P (c between any two food label class_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p, ask and be queried image collection SP={s_i| i=1,2 ..., N_sAt label group food set c={c_j| j=1,2 ... N_cMarkov MRF energy equation J (c) in }；

(4.5) utilize Iterative conditional modes algorithm ICM to minimize Markov MRF energy equation J (c), obtain i-th and be queried image s_iFood label class c_v；Reuse Iterative conditional modes algorithm ICM to minimize Markov MRF energy equation J (c) and obtain other that be queried in food image set SP and be queried the food label class of image, complete being queried the identification being queried image in food image set SP.

Present invention have the advantage that

1) due to the fact that use global characteristics descriptor is to describe image, it is to avoid the problem that its similarity cannot be judged because of food shapes indefinite form.Image is stored with the form of global characteristics descriptor, has saved memory space.

2) present invention uses the global characteristics descriptor of retrieval food image to explain and be queried image, can be readily extended to bigger be queried food image collection and tally set.

3) present invention passes through, to numerical statistic and the analysis being queried food image collection, to utilize Iterative conditional modes algorithm ICM to optimize the energy equation of Markov random field (MRF), improve computational efficiency.

Accompanying drawing explanation

Fig. 1 be the present invention realize general flow chart；

Fig. 2 is the sub-process figure of the directivity descriptor CEDD extracting color and edge in the present invention；

Fig. 3 is the Iterative conditional modes algorithm flow chart used in the present invention；

Fig. 4 be obtaining from the person's of being queried breakfast of being used of emulation experiment of the present invention be queried image；

Fig. 5 is the part retrieval image in the searching database of emulation experiment of the present invention.

Detailed description of the invention

With reference to Fig. 1, the present invention to realize step as follows:

Step 1, sets up searching database according to the food image of different food species, the retrieval image D from searching database_dMiddle extraction D_dN kind feature descriptorIndex building file, wherein D_dRepresent the d retrieval image, d=1,2 ..., N_d, N_dRepresent and retrieve total number of images, k=1,2 ... N, N represent the sum of the feature descriptor of use.

The main characteristics of image descriptor used in the present inventionIncluding the color layout descriptors CLD in directivity rectangular histogram descriptor BTDH, fuzzy color and the Texture similarity descriptor FCTH of directivity descriptor CEDD, brightness and texture at the color in compact descriptor and edge and Multimedia Content Description Interface MPEG-7 visual standards, edge histogram descriptor EHD, Scalable Color Descriptor SCD, the extraction step of these several characteristics of image descriptors is as follows:

(1.1) extraction of the directivity descriptor CEDD feature at color and edge:

Reference Fig. 2, being implemented as follows of this step:

(1.1a) first retrieval image is divided into some piecemeals, more each piecemeal is divided into 4 sub-blocks；

(1.1b) each sub-block is extracted texture information: in YIQ color space, i.e. brightness-tone-saturation space, calculates the gray value of each pixel, obtains the average gray value of the pixel of each sub-block；Again by each sub-block through 5 digital filters filter, obtain edge histogram；Judge texture information classification according to edge histogram, obtain a 6-bin rectangular histogram；

(1.1c) each sub-block is extracted colouring information: each sub-block is transformed into HSV color space, i.e. channel hue-saturation-brightness color space, and calculates the meansigma methods of each passage of HSV color space；After 10-bins blur filter Filtration Filtration, according to 10 colors that are worth to of channel tone H, then by 24-bins blur filter filter after, according to saturation S and the regional determination of brightness V, channel tone H is classified again, obtains the rectangular histogram of a 24-bin；The rectangular histogram of 24-bin is added separately in the texture classification belonging to this sub-block, obtains the rectangular histogram of a 144-bin；

(1.1d) all piecemeals of retrieval image are repeated above-mentioned (1.1b)-(1.1.c) step, obtain the rectangular histogram of whole retrieval image, again rectangular histogram is normalized, obtains retrieving color and the directivity descriptor CEDD at edge of image；

(1.2) the directivity rectangular histogram descriptor BTDH of brightness and texture extracts:

(1.2a) first retrieval image is divided into some piecemeals, more each piecemeal is divided into 4 sub-blocks；

(1.2b) each sub-block is extracted texture information: each image subblock obtains three texel through Harr wavelet transformation, through Fuzzy Correlation system by this partition to 8-bin Texture similarity；

(1.2c) each sub-block is extracted colouring information: each sub-block is transformed into HSV color space, i.e. channel hue-saturation-brightness color space；After 24-bin blur filter Filtration Filtration, it is worth to a 24-bin rectangular histogram according to brightness V, the rectangular histogram of 24-bin is added separately in the texture classification belonging to this sub-block, obtains the rectangular histogram of sub-block；

(1.2d) all piecemeals of retrieval image are repeated above-mentioned (1.2b)-(1.2c) step, obtain the rectangular histogram of whole retrieval image, again rectangular histogram is normalized, obtains retrieving brightness and the directivity rectangular histogram descriptor BTDH of texture of image；

(1.3) fuzzy color and Texture similarity descriptor FCTH extract:

(1.3a) first retrieval image is divided into some piecemeals, more each piecemeal is divided into 4 sub-blocks；

(1.3b) each sub-block is extracted texture information: each image subblock obtains three texel through Harr wavelet transformation, through Fuzzy Correlation system by this partition to 8-bin Texture similarity；

(1.3c) each sub-block is extracted colouring information: each sub-block is transformed into hsv color space；Channel tone H, saturation S and the brightness V of each sub-block, through 10-bin Fuzzy Correlation system, obtain the rectangular histogram of 10-bin；By 24-bin Fuzzy Correlation system, original each channel tone H is divided into three tone H1 that the depth is different again, H2, H3, brightness V is changed into two fuzzy intervals, draw the rectangular histogram of a 24-bin, again the rectangular histogram of 24-bin is added separately in the Texture similarity belonging to this sub-block, obtains the rectangular histogram of sub-block；

(1.3d) all piecemeals of image are performed above-mentioned (1.3b)-(1.3c) step, obtain the rectangular histogram of whole image, by rectangular histogram normalization, obtain retrieving fuzzy color and the Texture similarity descriptor FCTH of image；

(1.4) edge histogram descriptor EDH extracts:

(1.4a) first retrieval image is divided into some piecemeals, more each piecemeal is divided into 4 sub-blocks；

(1.4b) each sub-block is extracted marginal information: in YIQ color space, calculate the gray value of each pixel, obtain the average gray value of the pixel of each sub-block；Each sub-block through 5 digital filters filter, obtains edge histogram again, judges that texture information classification obtains the rectangular histogram of a 5-bin according to edge histogram；

(1.4c) all piecemeals of image are performed above-mentioned (1.4b) step, obtain the rectangular histogram of whole image, by rectangular histogram normalization, obtain retrieving the edge histogram descriptor EDH of image.

(1.5) color layout descriptors CLD extracts:

(1.5a) map an image to YCbCr color space, i.e. Luma-Blue concentration excursion amount-red-color concentration side-play amount space, and divide the image into 64 piecemeals；

(1.5b) each piecemeal is extracted colouring information: calculate the meansigma methods of each color component of all pixels in each piecemeal；Each color component meansigma methods in piecemeal is carried out two-dimension discrete cosine transform DCT, obtains a series of two-dimension discrete cosine transform DCT coefficient of each component；The word scanning carrying out the two-dimension discrete cosine transform DCT coefficient of each component and quantization, take out the low frequency component of respective two-dimension discrete cosine transform DCT, and these three groups of low frequency components constitute the color layout descriptors CLD of retrieval image block；

(1.5c) all retrieval image blocks are repeated above-mentioned (1.5b) step, obtain the color layout descriptors CLD of whole retrieval image；

(1.6) Scalable Color Descriptor SCD extracts:

(1.6a) first retrieval image is divided into some piecemeals, more each piecemeal is divided into 4 sub-blocks；

(1.6b) each sub-block is extracted colouring information: each sub-block is transformed into hsv color space；Hsv color space uniform is quantified as 256bins (rectangular histogram), then is 11bit by each histogram value non-uniform quantizing, obtain the Scalable Color Descriptor SCD of sub-block through Haar transform coding.

(1.6c) all retrieval image blocks are repeated above-mentioned (1.6b) step, obtain the Scalable Color Descriptor SCD of whole retrieval image；

(1.7) repeat each the retrieval image in searching database is carried out feature descriptor extraction；

(1.8) by said extracted to retrieval image feature descriptor store, constitute index file.

Step 2, extracts the feature descriptor being queried image

Set i-th and be queried image s_i, and extract s_iKth kind feature descriptorI=1,2 ..., N_s, N_sRepresent and be queried total number of images；

The feature descriptor classification retrieving image that the feature descriptor being queried image that this step is extracted extracts with step 1 is identical, i.e. includes the color layout descriptors CLD in directivity rectangular histogram descriptor BTDH, fuzzy color and the Texture similarity descriptor FCTH of directivity descriptor CEDD, brightness and the texture at the color in compact descriptor and edge and Multimedia Content Description Interface MPEG-7 visual standards, edge histogram descriptor EHD, Scalable Color Descriptor SCD.

Extracting method is identical with the extracting method of step 1, wherein:

The directivity descriptor CEDD of the color and edge that are queried image extracts process the most above-mentioned (1.1) step；

The directivity rectangular histogram descriptor BTDH of brightness and texture extracts process the most above-mentioned (1.2) step；

Fuzzy color and Texture similarity descriptor FCTH extract process the most above-mentioned (1.3) step；

Color layout descriptors CLD extracts process the most above-mentioned (1.4) step；

Edge histogram descriptor EHD extracts process the most above-mentioned (1.5) step；

Scalable Color Descriptor SCD extracts process the most above-mentioned (1.6) step.

Step 3, according to retrieval image and the feature descriptor being queried image, calculates i-th and is queried image s_iIn jth label class c_jLikelihood Score P (s in food_i|c_j), j=1,2 ..., N_c, N_cThe sum of the food label classification in expression searching database.

(3.1) according to retrieval image D_dFeature descriptorBe queried image s_iFeature descriptorThe characteristic distance of calculating kth kind feature descriptor between the two

With the characteristic distance of the feature descriptor between image in the present inventionThe similarity representing retrieval image and be queried image, and use Tanimoto coefficientRepresent the characteristic distance of feature descriptor between imageTanimoto coefficient is the least, and similarity is the least；

${T_{i,d}}^k=t(f_i^k,f_d^k)=\frac{{\left(f_i^k\right)}^T{f}_d^k}{{\left(f_i^k\right)}^T{f}_i^k+{\left(f_d^k\right)}^T{f}_d^k-{\left(f_i^k\right)}^T{f}_d^k}---<1>$

WhereinRepresent i-th query image s_iIn kth kind feature descriptor,Represent the d retrieval image D_dIn kth kind feature descriptor,It it is feature descriptorTransposition,It it is feature descriptorTransposition,RepresentWithBetween Tanimoto coefficient；

(3.2) the retrieval image in double counting searching database and i-th are queried image s_iThe characteristic distance of kth kind feature descriptor, and according to the retrieval image in characteristic distance descending searching database, selected characteristic distance is less than the distance threshold h set_kRetrieval image, pie graph image setThis distance threshold h_kSetting relevant with the size of searching database；

(3.3) according to the image set obtained in above-mentioned steps (3.2)Calculating is queried image s_iIn label class c_jThe likelihood probability of the kth kind feature descriptor in food

$P\left(f_i^k\right|c_j)=n(c_j,N_i^k)/n(c_j,D)---<2>$

Wherein D represents retrieval food image collection, and c is the set of food label class in searching database,Represent image setMiddle label class c_jThe quantity of food, n (c_j, D) and represent label class c in retrieval food image collection D_jThe quantity of food；

(3.4) double counting i-th is queried image s_iIn jth label class c_jThe likelihood probability of other feature descriptors in food

(3.5) it is queried image s according to i-th_iIn jth label class c_jThe likelihood probability of the N number of feature descriptor in foodI-th is asked to be queried image s_iIn jth label class c_jLikelihood Score P (s in food_i|c_j):

$P\left(s_i\right|c_j)=\frac{1}{N}\underset{k=1}{\overset{N}{\&Sigma;}}{w}_{k}P\left(f_i^k\right|c_j),---<3>$

Wherein w_kRepresent that i-th is queried image s_iThe standard deviation of likelihood probability of kth kind feature descriptor, computing formula is as follows:

$w_k=\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}\left(P\right(f_i^k|c_j)-\stackrel{\&OverBar;}{P\left(f_i^k|c\right)}),---<4>$

WhereinRepresent that i-th is queried image s_iKth kind feature descriptor in label class c={c_j| j=1,2 ... N_cThe meansigma methods of the likelihood probability in }, computing formula is as follows:

$\stackrel{\&OverBar;}{P\left(f_i^k\right|c)}=\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}P\left(f_i^k\right|c_j);---<5>$

(3.6) each during double counting is queried image collection SP is queried the image Likelihood Score { P (s at each food label apoplexy due to endogenous wind_i|c_j) | i=1,2 ..., N_s, j=1,2 ..., N_c}。

Step 4, builds and is queried food image set SP={s_i| i=1,2 ..., N_sFood label class set c={c in searching database_j| j=1,2 ..., N_cMarkov MRF energy equation J (c) formula in }, and by minimizing Markov MRF energy J (c), show that i-th is queried image s_iFood label class c identified_v, v=1,2 ... N_c。

According to the conditional probability being queried in the Likelihood Score of image food label apoplexy due to endogenous wind in searching database and the person's of being queried menu between food label class in the present invention, build Markov MRF energy theorem, food identification problem being converted into and seeks Markov MRF energy equation J (c) the formula minimum problems being queried image set in retrieval food label class set, its step is as follows:

(4.1) m-th label class c in the person's of being queried menu is sought_mFood is in the n-th label class c_nProbability of occurrence P (c under food Conditions_m|c_n):

P(c_m|c_n)=n (c_m,M_m,n)/n(c_n,G_n)<6>

Wherein n (c_n,G_n) represent G_nMiddle label class c_nThe quantity of food, G_nRepresent c in menu_nThe set of group food image；n(c_m,M_m,n) represent M_m,nMiddle label class c_mThe quantity of food, M_m,nRepresent label class c on menu_mFood and label class c_nThe situation that food occurs simultaneously, m, n=1,2 ..., N_p, N_pRepresent the food label class sum in the person's of being queried menu；

(4.2) the n-th label class c in the person's of being queried menu is sought_nFood is in m-th label class c_mProbability of occurrence P (c under food Conditions_n|c_m):

P(c_n|c_m)=n (c_n,M_m,n)/n(c_m,G_m)<7>

Wherein n (c_m,G_m) represent G_mMiddle label class c_mThe quantity of food, G_mRepresent c in menu_mThe set of group food image；n(c_n,M_m,n) represent M_m,nMiddle label class c_nThe quantity of food；

(4.3) conditional probability { P (c between any two food label class in the double counting person of being queried menu_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p}；

(4.4) it is queried the image Likelihood Score { P (s at food label apoplexy due to endogenous wind according to be queried in image collection SP_i|c_j) | i=1,2 ..., N_s, j=1,2 ..., N_cAnd the person's of being queried menu in conditional probability { P (c between any two food label class_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p, seek Markov MRF energy equation J (c) being queried image collection SP in food label class set c:

$J\left(c\right)=\underset{s_i\&Element;SP}{\&Sigma;}\&lsqb;1-w_{i}P(s_i,c_j)\&rsqb;+\mathrm{\&lambda;}\underset{(c_m,c_n)\&Element;A}{\&Sigma;}\&lsqb;1-\left(P\right(c_m|c_n)+P(c_n|c_m\left)\right)/2\&rsqb;---<8>$

Wherein A={ (c_m,c_n) | m, n=1,2 ..., N_pRepresenting the collection of food label class pair in the person's of being queried menu, λ is smoothing constant, w_iRepresent that i-th is queried image s_iThe standard deviation of Likelihood Score, computing formula is as follows:

$w_i=\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}\left(P\right(s_i|c_j)-\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}P(s_i|c_j\left)\right)---<9>$

(4.5) Iterative conditional modes algorithm ICM is utilized to minimize Markov MRF energy equation J (c), as it is shown on figure 3, obtain i-th to be queried image s_iFood label class c_v, step is as follows:

(4.5a) P (s is initialized_i|c_j),P(c_m|c_n), cycle-index Q=0 is set；

(4.5b) choose be different from currently be queried image be queried image, calculate Markov MRF energy J (c, s according to formula<8>_i)={ J (c_j,s_i) | j=1,2 ..., N_c}；

(4.5c) from Markov MRF energy J (c, s_iSelect energy minimum in), obtain food label class c of correspondence_v, it is i-th and is queried image s_iFood qualification category；

(4.5d) Q=Q+1 is made, if Q is < N_s, return to step (4.5b), otherwise, stop circulation, draw the food label class being queried image being queried in food image set SP, complete being queried the identification being queried image in food image set SP.

The effect of the present invention can be further illustrated by following experiment.

1. experimental subject

Experimental subject is the breakfast food image of the person of the being queried some day shown in Fig. 4, and the person's of being queried breakfast menu in a week is as shown in table 1:

Breakfast menu of table 1 person of being queried

2. experimental procedure

(2.1) set up searching database such as Fig. 5, extract the feature descriptor of retrieval image, index building file；

(2.2) it is queried image shown in input Fig. 4, from being queried image extraction feature descriptor, and calculates the characteristic distance of feature descriptor of itself and retrieval image；

(2.3) feature descriptor being queried food image is obtained shown in the likelihood probability result table 2 of food label apoplexy due to endogenous wind according to formula<2>；

Table 2

(2.4) according to formula<3>, obtain input is queried food image Likelihood Score in label group food, is P (s successively₁|c),P(s₂|c),P(s₃| c), as follows:

P(s₁| c)={ 0.2185,0.0700,0.1700,0.2684,0.0757,0.1974}

P(s₂| c)={ 0.1131,0.4716,0.0725,0.1497,0.1374,0.0556}

P(s₃| c)={ 0.0362,0.1757,0.0598,0.1415,0.5332,0.0536}

(2.5) according to table 1, formula<6>and formula<7>, obtain the conditional probability P (c between the label class in the person's of being queried breakfast menu sheet 1_m|c_n), P (c_n|c_m), such as table 3

Table 3

(2.6) use Iterative conditional modes algorithm ICM to minimize Markov MRF energy theorem, obtain following data:

J(c,s₁)={ 2.2902,5.9449,2.3388,5.7490,5.9392,2.3113}

J(c,s₂)={ 6.3207,1.5185,4.1906,6.2006,6.8626,4.2074}

J(c,s₃)={ 6.3977,6.8243,4.2033,6.2088,1.4569,4.2095}

The data wherein darkened are the minima in these group data.

3. experimental result:

Markov MRF energy-minimum from step (2.6), the label class obtaining corresponding input picture is: steamed bun, egg, milk, and the food species being queried image with Fig. 4 of input complies fully with.

Claims (6)

1. fast food based on markov random file recognition methods, including:

(1) searching database is set up according to the food image of different food species, the retrieval image D from searching database_dMiddle extraction D_dN kind feature descriptorAnd store it in index file, D_dRepresent the d retrieval image, d=1,2 ..., N_d, N_dRepresent and retrieve total number of images, k=1,2 ... N, N represent the sum of the feature descriptor of use；

(2) set i-th and be queried image s_i, and extract s_iKth kind feature descriptor f_i ^k, i=1,2 ..., N_s, N_sRepresent and be queried total number of images；

(3) according to retrieval image and the feature descriptor being queried image, calculate i-th and be queried image s_iIn jth label class c_jLikelihood Score P (s in food_i|c_j), j=1,2 ..., N_c, N_cThe sum of the food label classification in expression searching database；

(4) structure is queried food image set SP={s_i| i=1,2 ..., N_sFood label class set c={c in searching database_j| j=1,2 ..., N_cMarkov MRF energy equation J (c) formula in },, and by minimizing Markov MRF energy J (c), show that i-th is queried image s_iFood label class c identified_v, v=1,2 ... N_c:

(4.1) m-th label class c in the person's of being queried menu is calculated_mFood is in the n-th label class c_nProbability of occurrence P (c under food Conditions_m|c_n) n, m=1,2 ..., N_p, n ≠ m, N_pRepresent the sum of food label classification in the person's of being queried menu；

(4.2) the n-th label class c in the person's of being queried menu is sought_nFood is in m-th label class c_mProbability of occurrence P (c under food Conditions_n|c_m)；

(4.3) conditional probability { P (c between any two food label class in the double counting person of being queried menu_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p}；

(4.4) it is queried the image Likelihood Score { P (s at food label apoplexy due to endogenous wind according to be queried in image collection SP_i|c_j) | i=1,2 ..., N_s, j=1,2 ..., N_cAnd the person's of being queried menu in conditional probability { P (c between any two food label class_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p, ask and be queried image collection SP={s_i| i=1,2 ..., N_sAt label group food set c={c_j| j=1,2 ... N_cMarkov MRF energy equation J (c) in }；

(4.5) utilize Iterative conditional modes algorithm ICM to minimize Markov MRF energy equation J (c), obtain i-th and be queried image s_iFood label class c_v；Reuse Iterative conditional modes algorithm ICM to minimize Markov MRF energy equation J (c) and obtain other that be queried in food image set SP and be queried the food label class of image, complete being queried the identification being queried image in food image set SP.

Food recognition methods the most according to claim 1, the wherein step (1) retrieval image D from searching database_dMiddle extraction N kind feature descriptorSet up index file, carry out in accordance with the following steps:

(1.1) will retrieval image D_dIt is divided into some piecemeals, more each piecemeal is divided into 4 sub-blocks, and each sub-block is carried out texture, color and edge treated, obtain the rectangular histogram of piecemeal；Repeat all piecemeals are carried out texture, color and edge treated, obtain retrieving image D_dRectangular histogram, and to this retrieval image D_dRectangular histogram be normalized obtain retrieve image D_dN kind feature descriptor；

(1.2) repeat each the retrieval image in searching database is carried out feature descriptor extraction, and store, constitute index file.

Food recognition methods the most according to claim 1, extracts in wherein said step (2) and is queried image s_iFeature descriptor, be will to be queried image s_iIt is divided into some piecemeals, more each piecemeal is divided into 4 sub-blocks, and each sub-block is carried out texture, color and edge treated, obtain the rectangular histogram of piecemeal；Repeat all piecemeals are carried out texture, color and edge treated, obtain being queried image s_iRectangular histogram, then to being queried image s_iRectangular histogram be normalized and obtain being queried image s_iN kind feature descriptor.

Method the most according to claim 1, wherein according to retrieval image and the feature descriptor being queried image in step (3), calculates i-th and is queried image s_iIn jth label class c_jLikelihood Score P (s in food_i|c_j), carry out as follows:

(3.1) according to retrieval image D_dFeature descriptorBe queried image s_iFeature descriptor f_i ^k, the characteristic distance T of calculating kth kind feature descriptor between the two_i,d ^k；

${T_{i,d}}^k=t(f_i^k,f_d^k)=\frac{{\left(f_i^k\right)}^T{f}_d^k}{{\left(f_i^k\right)}^T{f}_i^k+{\left(f_d^k\right)}^T{f}_d^k-{\left(f_i^k\right)}^T{f}_d^k}---<1>$

Wherein f_i ^kRepresent i-th query image s_iIn kth kind feature descriptor,Represent the d retrieval image D_dIn kth kind feature descriptor, (f_i ^k)^TIt is feature descriptor f_i ^kTransposition,It it is feature descriptorTransposition；

(3.2) the retrieval image in double counting searching database and i-th are queried image s_iThe characteristic distance of kth kind feature descriptor, and according to the retrieval image in characteristic distance descending searching database, selected characteristic distance is less than setting threshold value h_kRetrieval image, pie graph image set

(3.3) basis and is queried image s in retrieval image set_iKth kind feature descriptor characteristic distance less than set threshold value h_kRetrieval image setCalculating is queried image s_iIn label class c_jLikelihood probability P (the f of the kth kind feature descriptor in food_i ^k|c_j)；

$P\left(f_i^k\right|c_j)=n(c_j,N_i^k)/n(c_j,D)---<2>$

Wherein D represents retrieval food image collection, and c is the set of food label class in searching database,Represent retrieval image setMiddle label class c_jThe quantity of food, n (c_j, D) and represent label class c in retrieval food image collection D_jThe quantity of food；

(3.4) double counting i-th is queried image s_iIn jth label class c_jLikelihood probability { P (the f of other feature descriptors in food_i ^k|c_j) | k=1,2 ..., N}；

(3.5) it is queried image s according to i-th_iIn jth label class c_jLikelihood probability { P (the f of the N number of feature descriptor in food_i ^k|c_j) | k=1,2 ..., N}, asks by i-th query image s_iIn jth label class c_jLikelihood Score P (s in food_i|c_j):

$P\left(s_i\right|c_j)=\frac{1}{N}\underset{k=1}{\overset{N}{\&Sigma;}}{w}_{k}P\left(f_i^k\right|c_j)---<3>$

Wherein w_kRepresent that i-th is queried image s_iThe standard deviation of likelihood probability of kth kind feature descriptor, computing formula is as follows:

$w_k=\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}\left(P\right(f_i^k|c_j)-\stackrel{\&OverBar;}{P\left(f_i^k|c\right)})---<4>$

WhereinRepresent that i-th is queried image s_iKth kind feature descriptor in label class c={c_j| j=1,2 ... N_cThe meansigma methods of the likelihood probability in }, computing formula is as follows:

$\stackrel{\&OverBar;}{P\left(f_i^k\right|c)}=\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}P\left(f_i^k\right|c_j)---<5>$

(3.6) each during double counting is queried image collection SP is queried the image Likelihood Score { P (s at each food label apoplexy due to endogenous wind_i|c_j) | i=1,2 ..., N_s, j=1,2 ..., N_c}。

Food recognition methods the most according to claim 1, wherein builds in step (4) and is queried food image set SP={s_i| i=1,2 ..., N_sFood label class set c={c in searching database_j| j=1,2 ..., N_cMarkov MRF energy equation J (c) formula in }, is carried out as follows:

(4.1) m-th label class c in the person's of being queried menu is sought_mFood is in the n-th label class c_nProbability of occurrence P (c under food Conditions_m|c_n):

P(c_m|c_n)=n (c_m,M_m,n)/n(c_n,G_n)<6>

Wherein n (c_n,G_n) represent G_nMiddle label class c_nThe quantity of food, G_nRepresent c in menu_nThe set of group food image；n(c_m,M_m,n) represent M_m,nMiddle label class c_mThe quantity of food, M_m,nRepresent label class c on menu_mFood and label class c_nThe situation that food occurs simultaneously, m, n=1,2 ..., N_p；

(4.2) the n-th label class c in the person's of being queried menu is sought_nFood is in m-th label class c_mProbability of occurrence P (c under food Conditions_n|c_m):

P(c_n|c_m)=n (c_n,M_m,n)/n(c_m,G_m)<7>

Wherein n (c_m,G_m) represent G_mMiddle label class c_mThe quantity of food, G_mRepresent c in menu_mThe set of group food image；n(c_n,M_m,n) represent M_m,nMiddle label class c_nThe quantity of food, M_m,nRepresent label class c on menu_mFood and label class c_nThe situation that food occurs simultaneously, m, n=1,2 ..., N_p；

(4.3) conditional probability { P (c between any two food label class in the double counting person of being queried menu_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p}；

(4.4) it is queried the image Likelihood Score { P (s at food label apoplexy due to endogenous wind according to be queried in image collection SP_i|c_j) | i=1,2 ..., N_s, j=1,2 ..., N_cAnd the person's of being queried menu in conditional probability { P (c between any two food label class_m|c_n),P(c_n|c_m) | m, n=1,2 ..., N_p, seek Markov MRF energy equation J (c) being queried image collection SP in food label class set c:

$J\left(c\right)=\underset{s_i\&Element;SP}{\&Sigma;}\&lsqb;1-w_{i}P(s_i,c_j)\&rsqb;+\mathrm{\&lambda;}\underset{(c_m,c_n)\&Element;A}{\&Sigma;}\&lsqb;1-\left(P\right(c_m|c_n)+P(c_n|c_m\left)\right)/2\&rsqb;---<8>$

Wherein, A={ (c_m,c_n) | m, n=1,2 ..., N_pRepresenting the collection of food label classification pair in the person's of being queried menu, λ is smoothing constant, w_iRepresent that i-th is queried image s_iThe standard deviation of Likelihood Score, computing formula is as follows:

$w_i=\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}\left(P\right(s_i|c_j)-\frac{1}{N_c}\underset{j=1}{\overset{N_c}{\&Sigma;}}P(s_i|c_j\left)\right)---<9>$

Food recognition methods the most according to claim 1, wherein utilizes Iterative conditional modes algorithm ICM to minimize Markov MRF energy equation J (c), obtains i-th and be queried image s in step (4)_iFood label class c_v, step is as follows:

(4.5a) P (s is initialized_i|c_j),P(c_m|c_n), cycle-index Q=0 is set；

(4.5b) choose be different from currently be queried image be queried image, calculate Markov MRF energy J (c, s according to formula (8)_i)={ J (c_j,s_i) | j=1,2 ..., N_c}；

(4.5c) from Markov MRF energy J (c, s_iSelect energy minimum in), obtain food label class c of correspondence_v, it is i-th and is queried image s_iFood qualification category；

(4.5d) Q=Q+1 is made, if Q is < N_s, return to (4.5b) step, otherwise stop circulation, draw the food label class being queried image being queried in food image set SP, complete being queried the identification of image in food image set SP.