CN101153851B - Apple detection classification method based on machine vision - Google Patents

Abstract

The present invention relates to an apple detection grading method based on machine vision, including the following steps: an apple rolls continuously to pass a collection zone and at least three different surface images are collected in succession so that the images cover most surface area of the apple; the area of the obtained images is taken as circular area; moreover, the mid diameter corresponding to the circular area is calculated and is taken as the size parameter of the apple; meanwhile, the average value of the surface red area proportion, the surface average chroma and the surface coloring evenness of the apple are calculated to obtain the color parameter of the apple according to the obtained images; after background segmentation, gray processing, image sharpening and filtering denoising of the obtained images, the average gradient value of the images is calculated. The method can effectively reduce imaging linear error of scanner and is fast and accurate, thereby being suitable for various types of scanners; meanwhile, with low computation complexity, the method is easy to realize and suitable to be used in quick real-time grading of apple; moreover, the method realizes detection grading of five sorts of apples, thereby having wide application range.

Description

A kind of apple detection classification method based on machine vision

Technical field

The present invention relates to a kind of method of the quality detection and classification according to fruit, relate in particular to a kind of based on machine vision effect detection stage division.

Background technology

Apple is one of the fruit the most widely that is eaten in the world, also is important foreign trade fruit simultaneously.But the back is detected, classification technique falls behind owing to pluck, and at present a lot of apples are just handled directly listing without classification after harvesting, cause grade to mix, and are very different, influenced its commodity value, particularly lack competitiveness in the international market.

Apple adopt major technique link that the back commercialization handles comprise select, clean, waxing, classification and packing etc., wherein classification is the core link during commercialization is handled.The apple classification of China at present mainly still relies on and manually finishes, and the labour who needs is many, and it is powerful to work, and the result of classification is bigger because of laborer's individual difference difference, and the consistance of classification is relatively poor, and efficient is lower.The classification that utilizes machine vision to carry out machine vision has very big advantage, can realize single standard grading to apple, perhaps simultaneously to a plurality of standards, comprising that the exterior quality such as size, color, shape, surface imperfection of fruit is disposable carries out comprehensive classification.The objectivity of classification is strong, standard is stable, high conformity, efficient height, and noncontact do not have injury, thereby has good application prospects.

The vision classification of apple mainly is to carry out Digital Image Processing by the imagery exploitation computing machine to apple, extracts the apple eigenwert, and then differentiates the grade of fruit.Thereby the calculating of apple image processing techniques and eigenwert is the core technology of most critical in the whole vision classification, is directly connected to the feasibility and the accuracy of classification.

Summary of the invention

(1) technical matters that will solve

The objective of the invention is to overcome the deficiencies in the prior art, a kind of apple detection classification method based on machine vision is provided, this method is passed through the detection to size, color and three index parameters of texture of apple, and the classification nominal value in conjunction with setting carries out classification to apple.

(2) technical scheme

At above problem, it is a kind of based on machine vision effect detection stage division that the present invention proposes, and may further comprise the steps:

A kind of apple detection classification method based on machine vision, the detected parameters that wherein said apple detects comprises size, color and the texture of apple, it is characterized in that, said method comprising the steps of:

1) make the apple continuous rolling by pickup area, and continuous acquisition at least 3 width of cloth different surfaces images, make image cover most areas of apple surface;

2) area with the image obtained in the step 1) is considered as the area of a circle, and calculates the mean diameter of this area of a circle correspondence, and this mean diameter is the size parameter of apple;

3) calculate the mean value of the surperficial red sector ratio of apple, surperficial average chrominance and surface colour uniformity coefficient respectively according to the image of obtaining in the step 1), according to the surface color characteristics of different cultivars apple to above-mentioned three mean values, after being weighted COMPREHENSIVE CALCULATING, obtain the color parameter of apple;

Wherein, above-mentioned surperficial red sector ratio is meant the ratio of the shared whole fruit face of surperficial red area of apple,

The computing formula of above-mentioned surperficial average chrominance is:

$H=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{H}_i$

In the formula, the picture element number of apple in the n presentation video, H _iThe chromatic value of representing each picture element,

Above-mentioned surface colour uniformity coefficient has following two kinds of computing method:

(a) certain surface colour of apple is inhomogeneous, and when color was block the distribution, at first background segment was extracted the apple target; Utilize red area different then, find appropriate threshold, extract red area, and calculate its centre of form with the R value of green area; Ask the distance L of the red area centre of form and the whole apple centre of form, utilize the L value to judge apple surface uniform coloring degree, or (b) when certain surface of same apple differs big with the another one surface color, then ask for same apple respectively and clapped in three width of cloth images average chrominance H1, H2, the H3 of apple in every width of cloth image, ask parameter value dif again, formula is as follows, so with parameter d if differentiate apple surface uniform coloring degree: dif=max (| H ₁-H ₂|, | H ₁-H ₃|, | H ₂-H ₃|);

Description of drawings

4) image of obtaining in the step 1) is carried out after background segment, gray processing, sharpening processing and filtering and noise reduction handle, the texture that obtains apple highlights image, then, calculates the average gradient value of this image, and this Grad is the parametric texture of apple;

Wherein, described above-mentioned average gradient value obtains by the following method:

The image mid point (x, gradient y), direction be f (x, y) in the direction of this rate of change maximum, and its mould is G (f (x, y))) then equal f (x, maximum rate of change y), promptly

$G\left(f(x,y)\right)=\sqrt{{\left(\frac{\∂f}{\∂x}\right)}^2+{\left(\frac{\∂f}{\∂y}\right)}^2}$

Typical case gradient algorithm: G (f (x, y))=[(f (x, y)-f (x+1, y)) ²+ (f (x, y)-f (x, y+1)) ²] ^1/2Utilize typical gradient algorithm that image is carried out sharpening, calculate the Grad G (f (x of each pixel, y)), then image is carried out filtering and noise reduction, make texture highlight, after texture is clearly demonstrated, calculate the graded features value of this image: average gradient S, typical gradient algorithm is improved, obtain following gradient algorithm, G (f (x, y))=| f (x, y)-f (x+1, y) |+| f (x, y)-f (x, y+1) |, average gradient S then be Grad G (f's (x, y)) and with the ratio of apple diameter d:

$S=\frac{1}{d}\mathrm{\ΣG}\left(f(x,y)\right)$

Wherein d represents the diameter of apple;

5) one or more according in selected above-mentioned three detected parameters of the variety type of apple as the classification nominal parameters, and set corresponding classification nominal value range, after measured value and classification nominal value range compared, apple is carried out classification according to comparative result.

Preferably, the such scheme step 1) also can comprise: the original image of gathering is carried out the neighborhood filter preprocessing, extract its R channel image, it is carried out Threshold Segmentation, extract target image.

Embodiment

More preferably, in the such scheme step 3), the average chrominance maximum difference of each picture that obtains according to step 1) calculates the surface colour uniformity coefficient of apple.

(3) beneficial effect

When adopting method of the present invention that apple is carried out real-time classification, when the image resolution ratio to each apple collection was 195 * 195 pixels, hierarchical speed can reach 9 apples of per second.Can see from test findings: during for single yellow sweet apple of surface color and the comprehensive classification of Wang Lin apple, accuracy of identification reaches more than 95%, during the comprehensive classification of the red green alternate apple of surface color, identification accuracy reaches more than 91%, the computation complexity of the inventive method is low, be easy to realize, be suitable for the quick real-time grading of apple; And realized the detection classification of apple in 5, practical wide.

Fig. 1 is the apple sample that the surperficial red sector ratio of the embodiment of the invention 2 detects;

Fig. 2 is the hue distribution figure of four grades of the apple of the embodiment of the invention 2;

Fig. 3 is the accumulative total form and aspect frequency distribution plan of four grades of the apple corresponding with Fig. 2;

Fig. 4 is the process synoptic diagram that the embodiment of the invention 2 is calculated average chrominance H value;

Fig. 5 is the process synoptic diagram that the embodiment of the invention 2 detects painted inhomogeneous apple two centres of form;

Fig. 6 is the surperficial surface contrast figure that differs than Big Apple in the embodiment of the invention 2;

Fig. 7 is distinct fruit of the texture of the embodiment of the invention 3 and the not distinct fruit contrast of texture figure;

Fig. 8 is the image pretreatment process block diagram of the embodiment of the invention 3;

Fig. 9 is the image pretreating effect figure of the embodiment of the invention 3.

A kind of apple based on machine vision that the present invention proposes detects stage division and is described as follows in conjunction with the accompanying drawings and embodiments. Following embodiment only is used for explanation the present invention; and be not limitation of the present invention; the those of ordinary skill of relevant technical field; in the situation that does not break away from the spirit and scope of the present invention; can also make a variety of changes and modification; therefore all technical schemes that are equal to also belong to category of the present invention, and scope of patent protection of the present invention should be limited by each claim.

The external sort of fruit mainly comprises: size, shape, color, texture and defect characteristic. Weight and size can reflect the size of apple. The regularity of profile directly affects the attractive in appearance of apple, and then affects the market value of apple. Color is the important indicator of the exterior quality of fruit, high-quality apple general painted evenly and also color and luster good. The maturity of color reflection apple has reflected the inner qualities such as fruit ground pol, acidity and mouthfeel, scarlet or dark red should accounting for more than 50% on the ripe Apples face indirectly. The color characteristics intrinsic according to different cultivars, the different purposes of pressing fruit take the coloring degree of pericarp as Main Basis, can be determined its selling price. Texture is the distinctive feature of quartzy Fuji, and in quartzy Fuji, the apple mouthfeel of texture distinctness is better than the feint apple of texture, and market value is also higher, is very important so carry out the texture classification. The present invention is based on machine vision and mainly three features of size, color, texture of apple have been carried out classification research.

When utilizing machine vision to carry out the classification of apple, image is 2D signal, every width of cloth image of apple only contains the visual information on the direction, and classification need to utilize the maximum fruit footpath of fruit on the three-dimensional all directions in space to represent size, a sub-picture utilize the whole surperficial colouring information of fruit comprehensively to judge the color grade of fruit, so can't comprise the required full detail of classification. The present invention gathers the cubic graph picture to each apple respectively, require the different surface of each collection, and the surface energy of gathering for three times covers more than 95% of whole fruit surface.

The specific algorithm that detects classification according to apple size, color, texture is divided into 3 embodiment, and is as follows:

Embodiment 1

Each apple does not roll through not stopping in the process of pickup area, and by continuous acquisition to 3 width of cloth different surfaces images, cover more than 90% of whole fruit face, can more intactly reflect apple surface information, and the area of apple is an apple maximum cross section area in every width of cloth image.Because apple belongs to circular fruit kind, can be similar to apple in the image and regard circle as, bring round area formula into and calculate radius r.R is an apple maximum cross section radius, and three width of cloth images of each apple are asked for three r values, asks average again, promptly obtains the mean radius R of apple.Because in the standard GB 10654-89 of fresh apple exposure draft, size represents with maximum cross-sectional area diameter, so makes R multiply by 2 to obtain apple mean diameter D, and D as the size characteristic value, improved projected area method that Here it is.Concrete grammar is as follows:

Dynamically down apple is being detected classification, each apple is at the projected image S through collected 3 width of cloth different surfaces in the process of pickup area ₁, S ₂And S ₃Bring round area formula into and calculate radius r ₁, r ₂And r ₃, as shown in Equation (1).

$r_i=\sqrt{s_i/\mathrm{\π}},i=\mathrm{1,2,3}---\left(1\right)$

r _iFor the fruit radius of obtaining by three projected areas, ask mean radius R again, as formula (2), diameter D equals 2 times of R so, and replaces the apple mean diameter to differentiate the size of apple with D, and computing formula is as (3).

$R=\frac{1}{3}\underset{i=1}{\overset{3}{\mathrm{\Σ}}}{r}_i---\left(2\right)$

D＝2×R (3)

Embodiment 2

Choose a large amount of apple sample, its surface color characteristic is analyzed, obtain three surface color characteristic parameters: surperficial red sector ratio (ratio of the shared whole fruit face of surperficial red area), surperficial average chrominance, surface colour uniformity coefficient.Introduce the computing method of these three characteristic parameters below respectively.

The detection method of surface red sector ratio is as follows:

Be illustrated in figure 1 as 4 apple sample representing four grades.Utilize the HIS color model that it is carried out the color characteristics analysis.Calculation procedure is: at first, image segmentation is extracted the apple target; Then, the RGB color model is converted to the HIS color model; At last, obtain the chromatic value of each picture element in the target image, represented the colourity of four grade apple colors to distribute as shown in Figure 2 respectively.

Can get the red fuji apple chromaticity range from Fig. 2 about 0 °～100 °, with reducing of degree of staining, the form and aspect curve is offset to the right.Excellent red fuji apple chromatic value concentrates on about 0 °～25 °; First-class red fuji apple chromatic value concentrates on about 15 °～40 °; Second-class red fuji apple chromatic value concentrates on about 30 °～65 °, and the relatively dispersion that distributes; Concentrate on about 60 °～80 ° Deng outer red fuji apple chromatic value.But the hue distribution scope of each grade apple image has lap, is difficult to the form and aspect threshold value that finds differentiation at different levels.According to hue distribution Fig. 2, calculate frequency accumulative total under each hue value (frequency that promptly is not more than this chromatic value add up and), obtain corresponding form and aspect accumulative frequency and distribute as Fig. 3.As shown in Figure 3, along with apple is green by red stain, its form and aspect accumulative frequency figure obviously moves to right, and the curve steepness diminishes.So pairing accumulative frequency under the suitable hue value (being surperficial red sector ratio) as eigenwert, can come four grade distinguishings.

The detection method of surface average chrominance:

Apple surface for kinds such as yellow banana, Wang Lin is painted single, mostly is green or yellow green.The average chrominance that only needs to extract its surface color can be carried out color grading.As shown in Figure 4, under the detection of dynamic condition, obtain the apple image, again each width of cloth image is carried out image segmentation, the RGB color model is transformed into the HIS color model, the last chromatic value of under the HIS color model, obtaining each picture element in the target image, and then calculate the average chrominance value H of whole apple, suc as formula (1):

$H=\frac{1}{n}\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{H}_i,---\left(1\right)$

In the formula, the picture element number of apple target in the n presentation video, Hi represents the chromatic value of each picture element.

The detection method of surface colour uniformity coefficient:

A large amount of apple surface color characteristics is analyzed and can be got, and surface colour is inhomogeneous to be divided into two kinds of situations.

First kind of situation: certain surface colour of apple is inhomogeneous, and color is block and distributes, and as first apple among Fig. 5, at first background segment is extracted the apple target; Utilize red area different then, find appropriate threshold, extract red area, and calculate its centre of form with the R value of green area; Ask the distance L of the red area centre of form and the whole apple centre of form, utilize the L value to judge apple surface uniform coloring degree.As apple among Fig. 5, red area centre of form coordinate is (89,83), and whole apple centre of form coordinate is (97,86).And then to calculate 2 distances be 8.5 picture elements.

Second kind of situation: certain surface of same apple differs bigger with the another one surface color, as apple a among Fig. 6 and apple b, is the different surfaces of same apple, and obviously color distinction is very big.This problem is asked for same apple respectively and is clapped in three width of cloth images average chrominance H1, H2, the H3 of apple in every width of cloth image, asks parameter value dif again, and formula is shown in (2).And then differentiate apple surface uniform coloring degree with parameter d if.

dif＝max(|H ₁-H ₂|，|H ₁-H ₃|，|H ₂-H ₃|) (2)

Embodiment 3

Show that as Fig. 7 apple 1 and 2 background colors are yellowish, there are a large amount of textures on the surface, taste and sweet mouthfeel, and the market price is higher, is the distinct A level fruit of texture; It is red and green that apple 3 and 4 background colors are respectively, and surface color becomes block and distributes, and mouthfeel is taken second place, and the market price is lower, is the not distinct B level fruit of texture.

For ease of the following describes, Fig. 7 on the correspondence, wherein apple 3 surfaces are almost red entirely, and apple similar with it is defined as full haw, otherwise are non-full haw.

The Flame Image Process FB(flow block) as shown in Figure 8, method is described below:

1) adopts the neighborhood filtering method that original image is removed noise, set appropriate threshold and carry out image segmentation, extract the apple target;

2) by a large amount of tests as can be known, full haw and background color are difficult to cut apart with single Color Channel, need separately it to be separated to carry out Flame Image Process.At this, be in conjunction with R and B channel information proposition separation algorithm: the average absolute D of the difference of the R value of all pixels and B value in the computed image, promptly

$D=\frac{1}{N}\underset{i=1}{\overset{N}{\mathrm{\Σ}}}|R-B|---\left(3\right)$

By a large amount of tests, finding the threshold value that can distinguish above two kinds of apples is 55, and the D value is full haw greater than 55, otherwise is all the other class apples.Non-full haw and full haw obtain corresponding gray-scale map according to formula L=(3.5G-1.5R-B) and L=(R-G) respectively, and wherein L is a gray-scale value.

3) because texture region is positioned at the place of gray scale sudden change, can extract with grey scale difference.Because the apple surface texture usually has any direction in piece image, therefore choose isotropic sharpening method: gradient method.As Fig. 5-3 mid point (x, gradient y), direction be f (x, y) in the direction of this rate of change maximum, and its mould is G (f (x, y))) then equal f (x, maximum rate of change y), promptly

$G\left(f(x,y)\right)=\sqrt{{\left(\frac{\∂f}{\∂x}\right)}^2+{\left(\frac{\∂f}{\∂y}\right)}^2}---\left(4\right)$

Typical case's gradient algorithm:

G(f(x，y))＝[(f(x，y)-f(x+1，y)) ²+(f(x，y)-f(x，y+1)) ²] ^1/2。

Utilize typical gradient algorithm that image is carried out sharpening, (Grad is changed to black greater than 50 pixel for f (x, y)), setting threshold 50, and all the other points are changed to white to calculate the Grad G of each pixel.Doublely image is carried out the gradient method sharpening handle.Then image is carried out filtering and noise reduction, make texture highlight, arthmetic statement is: pixel is white in image, calculate its 8 gray values of pixel points sum sum on every side, if sum is shown as black to this pixel less than 1020 (255 * 4), otherwise display white still; Pixel is a black in image, calculates its 8 gray values of pixel points sum sum on every side, if sum is shown as white to this pixel, otherwise still shows black greater than 1785 (255 * 7).Apple 2 among Fig. 7 and apple 3 effect after above-mentioned processing is corresponding shown in Figure 9 respectively, and texture just highlights out like this.

After texture is clearly demonstrated, calculate the graded features value of this image: average gradient S.Typical gradient algorithm is improved, and raising speed obtains following gradient algorithm, suc as formula (6).Average gradient S then be Grad G (f's (x, y)) and with the ratio of apple diameter d, suc as formula (7).

G(f(x，y))＝|f(x，y)-f(x+1，y)|+|f(x，y)-f(x，y+1)| (6)

$S=\frac{1}{d}\mathrm{\ΣG}\left(f(x,y)\right)---\left(7\right)$

Wherein d represents the diameter of apple.

At last,, choose appropriate threshold, apple is divided into distinct fruit of texture and the not distinct fruit of texture according to textural characteristics for the eigenwert average gradient.

Calculate after above three surface characteristics, obtain big or small rank, color rank and the texture rank of apple, obtain the comprehensive rank of apple again by the weighted value of various combination, finish the comprehensive classification of apple.

According to the mode of operation of above-mentioned enforcement 1,2,3, high-class product, Grade A, goods of inferior quality and the off standard of apple are represented with A, B, C and D respectively.Draw the classification accuracy of A level apple as the formula (8).

$p\left(A\right)=\frac{\underset{i=1}{\overset{5}{\mathrm{\Σ}}}|A_i-A|}{n\×A}---\left(8\right)$

In the following formula, the apple when p (A) expression detects high-class product detects accuracy; Ai represents to test in the high-class product detection at every turn and examines the apple number; A represents manual detection high-class product apple number; N represents test number (TN); The overall classification accuracy of P representative test.

Because five kinds of apple surface feature differences, wherein quartzy Fuji surface color is red green alternate, and the color distribution complexity needs according to size, color, texture, comprehensive classification; Kudo Fuji, yellow banana and red banana need by size, color, comprehensive classification; And the Wang Lin color is single, mostly is green, only needs size classification to get final product.Five kinds of apple classification test data are as follows:

Table 1 size fractionation test findings

Table 2 color grading test findings

Table 3 texture classification test result

The comprehensive classification test findings of table 4

The present invention is 195 * 195 pixels to the image resolution ratio of each fruit collection when the actual classification in real time that is applied to apple, and hierarchical speed can reach 9 apples of per second.Can get from test findings: 1) when yellow sweet apple that surface color is single and the comprehensive classification of Wang Lin apple, accuracy of identification reaches more than 95%.2) during the comprehensive classification of the red green alternate apple of surface color, identification accuracy reaches more than 91%.

Claims (2)

1. apple detection classification method based on machine vision, the detected parameters that wherein said apple detects comprises size, color and the texture of apple, it is characterized in that, said method comprising the steps of:

1) make the apple continuous rolling by pickup area, and continuous acquisition at least 3 width of cloth different surfaces images, make image cover most areas of apple surface;

2) area with the image obtained in the step 1) is considered as the area of a circle, and calculates the mean diameter of this area of a circle correspondence, and this mean diameter is the size parameter of apple;

3) calculate the mean value of the surperficial red sector ratio of apple, surperficial average chrominance and surface colour uniformity coefficient respectively according to the image of obtaining in the step 1), according to the surface color characteristics of different cultivars apple to above-mentioned three mean values, after being weighted COMPREHENSIVE CALCULATING, obtain the color parameter of apple;

Wherein, above-mentioned surperficial red sector ratio is meant the ratio of the shared whole fruit face of surperficial red area of apple,

The computing formula of above-mentioned surperficial average chrominance is:

In the formula, the picture element number of apple in the n presentation video, H _iThe chromatic value of representing each picture element,

Above-mentioned surface colour uniformity coefficient has following two kinds of computing method:

(a) certain surface colour of apple is inhomogeneous, and when color was block the distribution, at first background segment was extracted the apple target; Utilize red area different then, find appropriate threshold, extract red area, and calculate its centre of form with the R value of green area; Ask the distance L of the red area centre of form and the whole apple centre of form, utilize the L value to judge apple surface uniform coloring degree, or (b) when certain surface of same apple differs big with the another one surface color, then ask for same apple respectively and clapped in three width of cloth images average chrominance H of apple in every width of cloth image ₁, H ₂, H ₃, ask parameter value dif again, formula is as follows, and then with parameter d if differentiate apple surface uniform coloring degree: dif=max (| H ₁-H ₂|, | H ₁-H ₃|, | H ₂-H ₃|);

4) image of obtaining in the step 1) is carried out after background segment, gray processing, sharpening processing and filtering and noise reduction handle, the texture that obtains apple highlights image, then, calculates the average gradient value of this image, and this Grad is the parametric texture of apple;

Wherein, described above-mentioned average gradient value obtains by the following method:

The image mid point (x, gradient y), direction be f (x, y) in the direction of this rate of change maximum, and its mould be G (f (x, y)) then equal f (x, maximum rate of change y), promptly

Typical case gradient algorithm: G (f (x, y))=[(f (x, y)-f (x+1, y)) ²+ (f (x, y)-f (x, y+1)) ²] ^1/2,

Utilize typical gradient algorithm that image is carried out sharpening, calculate the Grad G (f (x of each pixel, y)), then image is carried out filtering and noise reduction, make texture highlight, after texture is clearly demonstrated, calculate the graded features value of this image: average gradient S, typical gradient algorithm is improved, obtain following gradient algorithm, and G (f (x, y))=| f (x, y)-f (x+1, y) |+| f (x, y)-f (x, y+1) |, average gradient value S then be Grad G (f's (x, y)) and with the ratio of apple diameter d:

Wherein d represents the diameter of apple;

5) one or more according in selected above-mentioned three detected parameters of the variety type of apple as the classification nominal parameters, and set corresponding classification nominal value range, after measured value and classification nominal value range compared, apple is carried out classification according to comparative result.

2. apple detection classification method as claimed in claim 1 is characterized in that also comprising in described step 1): the original image of gathering is carried out the neighborhood filter preprocessing, extract its R channel image, it is carried out Threshold Segmentation, extract target image.

Images