CN108875747B

CN108875747B - Machine vision-based imperfect wheat grain identification method

Info

Publication number: CN108875747B
Application number: CN201810616422.XA
Authority: CN
Inventors: 何小海; 王周璞; 吴小强; 卿粼波; 滕奇志; 王正勇; 吴晓红
Original assignee: Sichuan University
Current assignee: Sichuan University
Priority date: 2018-06-15
Filing date: 2018-06-15
Publication date: 2021-10-15
Anticipated expiration: 2038-06-15
Also published as: CN108875747A

Abstract

The invention discloses a machine vision-based wheat imperfect grain identification method. The method comprises the following steps: collecting images of the upper surface and the lower surface of imperfect wheat grains in batches through collection equipment, preprocessing the images, registering the images after cutting boundaries, and removing image backgrounds; segmenting the preprocessed original image, segmenting the wheat images acquired in batch into single-grain images, wherein the segmentation is divided into two processes of rough segmentation and fine segmentation, and the rough segmentation roughly segments the whole registered image into single grains; the fine segmentation removes the background in the single-grain wheat image after the coarse segmentation, so that the interference of the background on wheat recognition is avoided; and (4) processing the wheat image according to the method, extracting the wheat characteristic, and identifying. The recognition method described by the invention has the advantages of short model training time, low dependence on the quality of the images collected by the front-end collecting equipment, mass recognition, high recognition speed and the like, and is beneficial to the practical application of wheat grain collection sites.

Description

Machine vision-based imperfect wheat grain identification method

Technical Field

The invention designs a machine vision-based imperfect wheat grain identification method. In particular to a method for identifying imperfect wheat grains based on self-adaptive wormhole positioning and spectral feature extraction.

Background

Wheat is one of three grain crops in China, the area and the yield of the wheat all account for about 1/4 of the grain crops in China, and the wheat yield and the wheat quality play a vital role in stabilizing domestic grain supply and guaranteeing grain safety. An important index for measuring the quality of wheat is the proportion of imperfect grains. Imperfect wheat grains do not refer to wheat that is completely worthless, and imperfect grains are primarily grains that represent damage. According to the national standard, the imperfect wheat grains mainly include broken grains, worm-forming grains, disease-forming grains (gibberellic disease grains and black embryo grains) and germinating grains. Imperfect wheat, once collected in grain depots, can have very serious consequences. Therefore, wheat quality control work is particularly important.

At present, the traditional manual detection method is adopted by the national food management and control department during food collection, and the method has the defects of high working strength, high requirement on professional skills of technicians, slow manual detection speed, certain human subjective factors and the like. In recent years, experts at home and abroad continuously focus on the research in the field, and a plurality of new detection methods are successively provided so as to realize the automation of the detection of the imperfect grains of wheat. Such as a method of detection using sound science, a detection method based on hyperspectral technology, and a detection method based on a Convolutional Neural Network (CNN) model. Although these methods can achieve the function of automated detection, there are still many disadvantages. For example, a microphone is used to collect the sound emitted by the vibration of wheat by using a sound science method, and the noise is very easily interfered by environmental sound and noise; the detection method based on the hyperspectral technology has the advantages that the image acquisition equipment is expensive, and the mixed detection of various imperfect grain varieties cannot be realized; the deep learning detection method based on the CNN model has the advantages of large required sample amount and long identification time.

Disclosure of Invention

The invention aims to solve the problems and provides a method for identifying imperfect wheat grains based on machine vision.

The invention realizes the purpose through the following technical scheme:

a machine vision-based wheat imperfect grain identification method comprises the following steps:

(1) the method comprises the steps of collecting images of the upper surface and the lower surface of imperfect wheat grains in batches through collection equipment, preprocessing the images, registering the images after cutting boundaries, and removing image backgrounds by utilizing series of operations such as graying and the like.

(2) Segmenting the preprocessed original image, changing the wheat images acquired in batch into single-grain images, and dividing the segmentation into two processes of rough segmentation and fine segmentation; roughly dividing the whole registered image into single wheat grains; the fine segmentation removes the background in the single-grain wheat image after the coarse segmentation, and avoids the wheat recognition interference caused by the background.

(3) And (3) processing the wheat image according to the method, extracting the wheat characteristics, and identifying the wheat.

Drawings

FIG. 1 is a flow chart of a machine vision-based method for identifying imperfect grains in wheat

FIG. 1-1 flow chart of wheat grain image preprocessing

FIG. 2-1 wheat grain image Pre-segmentation mask

FIG. 2-2 upper and lower views of rough wheat grain segmentation

FIGS. 2-3 comparison of rough and fine wheat grain segmentation

FIGS. 2-4 are graphs of contrast enhancement results of wheat grain images

FIG. 3-1 adaptive wormhole feature extraction block diagram

FIG. 3-2 is a graph showing the effect of the treatment of the vermin

FIGS. 3-3 comparison of frequency spectra of germinated and normal grains

FIGS. 3-4 comparison of frequency domain segmentation of germinated and normal grains

Detailed Description

The invention is further described below with reference to the accompanying drawings:

in fig. 1, a machine vision-based method for identifying defective wheat grains includes the following steps:

(2) Segmenting the preprocessed original image, changing the wheat images acquired in batch into single-grain images, dividing into two processes of rough segmentation and fine segmentation, and roughly segmenting the whole registered image into single wheat grains by the rough segmentation; the fine segmentation removes the background in the single-grain wheat image after the coarse segmentation, and avoids the wheat recognition interference caused by the background.

Specifically, in the step (1), firstly, a front-end collection device is used to collect 200-300 wheat grain double-sided images in batch, and the wheat grain images are preprocessed, so as to improve the recognition rate of the wheat grains and establish a clear wheat grain image library, as shown in a wheat grain image preprocessing flow chart in fig. 1-1. After the acquired original images are obtained, in order to ensure the position correspondence of the upper and lower images, the upper and lower images need to be registered, a template matching algorithm is adopted as a registration algorithm, and the similarity degree of the two images is calculated by adopting a correlation coefficient matching method. The correlation coefficient matching method subtracts the average values of the image I (x, y) and the template T (x, y) to obtain I '(x, y) and T' (x, y), so that both images have no direct current component, and the matching result R (x, y) is defined as:

where T' (x, y) is defined as:

i' (x, y) is defined as:

in the step (2), a mask image 2-1 before wheat grain image segmentation is obtained firstly. The segmentation is divided into two processes of rough segmentation and fine segmentation, wherein the rough segmentation roughly segments the whole registered image into single wheat grains; the fine segmentation removes the background in the single-grain wheat image after the coarse segmentation, and avoids the interference of the background on wheat recognition. Firstly, obtaining masks of upper and lower images of wheat grains; secondly, solving a union set of masks of the upper and lower graphs; and finally, obtaining the minimum circumscribed rectangle of each outline through outline detection, and cutting according to the size of the minimum circumscribed rectangle to obtain a rough wheat grain segmentation image, as shown in an upper graph and a lower graph of the rough wheat grain segmentation of fig. 2-2.

Comparing the rough segmentation and the fine segmentation of the wheat grains with those shown in fig. 2-3, the rough segmentation step can obtain a rough segmentation image of the wheat grains, and then the fine segmentation is performed on the wheat grains to remove the background of the wheat grain image. The wheat grain image displayed by the method has a pure and uncomplicated background, so that the method firstly selects a seed node, secondly selects an area communicated with the seed node and fills the communicated area by adopting specified colors, and the method is used for realizing fine segmentation. Here, the seed node is selected as the position of the roughly segmented wheat grain image (10, 10).

As shown in fig. 2-4, after obtaining the refined wheat grain segmentation image, image enhancement needs to be performed on the wheat grain image in order to extract more effective features, and an improved algorithm for limiting contrast adaptive histogram equalization is proposed. The specific process is as follows:

firstly, pixels of RGB three channels are counted in a histogram, then contrast-limiting adaptive histogram equalization processing is carried out, and then nonlinear mapping is carried out on image pixel points. The following expression maps the image gray-scale value i to T_iAnd the value of b is an enhanced strength parameter. The result of such a mapping process is that pixels that appear too dark in the image are enhanced more, while those that are themselves brighter are enhanced less.

T_i＝MIN(255,MAX(0,i+sin(PI*i/255.0f)*b))

And (4) in the step (3), performing wheat feature extraction and identification operation. By observing the images of the wheat grains, the differences of the wheat grains in three aspects of color, shape and texture can be known, which is also a feature generally adopted at present. The invention is characterized in that a wheat feature extraction and identification method based on self-adaptive wormhole positioning and frequency spectrum features is provided.

As shown in a self-adaptive wormhole feature extraction block diagram 3-1, on the basis of obtaining an optimal segmentation threshold, the position of the wormhole can be well self-adaptively positioned and the area of the wormhole can be quantized for each wheat image, and the wormhole is taken as a feature parameter for extraction. To illustrate the adaptive wormhole feature extraction method, the following formula variables need to be declared: the wheat image G (x, y) is w x h, the segmentation threshold of the wheat and the background color is marked as t, and the pixel less than the threshold tThe value is the target color, and the number is marked as G₀With a weighted mean gray value of c₀(ii) a The pixel values above the threshold t are background colors, the number of which is denoted G₁With a weighted mean gray value of c₁. Firstly, an optimal segmentation threshold is adaptively found for each wheat image, and the following formula can be obtained:

c＝b₀*c₀+b₁*c₁

t＝b₀*(c₀-c)²+b₁*(c₁-c)²

wherein, b₀The pixel value less than the threshold value t accounts for the proportion of the whole image; b₁The pixel value larger than the threshold value t accounts for the proportion of the whole image; c is the average gray value of the whole image; and obtaining the maximum t value after iteration, namely the optimal segmentation threshold value.

As shown in the fig. 3-2 of the effect of processing the worm grains, after the worm holes are positioned through the steps, the pixel value larger than the threshold is 255 and the rest is 0 by using a threshold segmentation method, so that the positions of the worm holes of the worm-eaten wheat are accurately positioned.

As shown in the comparison of frequency spectrums of the germinated grains and normal grains in the figure 3-3, the germinated grains have the characteristics of inconsistent sizes of bud tips of embryo parts, inconsistent emergence degrees of the bud tips and the like in a spatial domain and are easily identified as normal grains and broken grains by mistake. The invention finds that the germinated grains are obviously different from normal grains and broken grains in the shape and gray value transition of the wheat embryo part, and the difference is more obvious in the frequency domain.

The frequency domain division of the germinated grains and the normal grains is compared with that shown in FIGS. 3-4, and by utilizing the difference between the germinated grains and the normal grains in the frequency domain, characteristic parameters of entropy, energy, contrast and inverse difference moment of the spectrogram in the directions of 0 degree, 45 degrees, 90 degrees and 135 degrees are extracted. We can obtain the following formula:

wherein, Ent is an entropy value of the image, Energy is an Energy value of the image, and Contrast is a Contrast of the image. From the above description, the following feature extraction data table based on spectral features can be obtained:

TABLE 1 characteristic data table based on spectral characteristics

Experiments can show that the method described in the patent has obvious effect on identifying imperfect wheat grains, and the detailed experimental effect is shown in table 2:

TABLE 2 hybrid class identification results

Claims

1. A machine vision-based wheat imperfect grain identification method is characterized by comprising the following steps:

(1) acquiring 200-shot 300-shot double-sided images in batch by using front-end acquisition equipment, registering upper and lower images after acquiring acquired images, and subtracting respective average values from an image I (x, y) and a template T (x, y) to obtain I '(x, y) and T' (x, y), so that the two images have no direct-current component, and the matching result R (x, y) is defined as:

where T' (x, y) is defined as:

i' (x, y) is defined as:

wherein w represents the width of the image and h represents the height of the image;

(2) preprocessing the registered image in the step (1), removing the image background by using a series of operations such as graying and the like to obtain a preprocessed wheat original image, and enhancing the image of the wheat original image, wherein an improved method for limiting contrast and adapting to histogram equalization is provided and is expressed based on the following formula:

where i denotes the image grey scale value, T_iRepresenting the mapped gray value, b representing the enhanced intensity;

(3) according to the result of the step (2), a method for searching the optimal wormhole segmentation threshold value based on self-adaptive iteration of single-grain wheat is provided, the original segmentation image and the graphic layer picture of the single-grain wheat are obtained through rough segmentation, and the background is removed through fine segmentation on the basis of the rough segmentation; roughly dividing the whole registered image into single wheat grains, firstly solving masks of upper and lower images of the wheat grains, secondly solving a union set of the masks of the upper and lower images, finally obtaining a minimum external rectangle of each outline through outline detection, and cutting according to the size of the minimum external rectangle to obtain a roughly divided image of the wheat grains; the fine segmentation removes the background in the single-kernel image after the rough segmentation, the fine segmentation firstly sets a seed node A (10,10) for each image, pixel filling is carried out in sequence according to the connected domain of the current node from the seed node, and after the filling is finished, the image is enhanced by adopting the improved method for limiting the contrast ratio self-adaptive histogram equalization provided by the step (2);

(4) according to the wheat single grain fine segmentation image and the pattern layer image obtained in the step (3), a feature extraction method for searching an optimal wormhole segmentation threshold and multi-angle frequency spectrum characteristics based on self-adaptive iteration for the respective living insect grains and the germinating grains is provided on the basis of traditional feature extraction, firstly, a threshold segmentation method is adopted for each living insect grain image to find a wormhole position T, and the wormhole position features are quantized into an area matrix (T) at the position T;

for the insect grain, set wheat image G (x, y), size w h, wheat and background color segmentation threshold value as t, the pixel value less than threshold value t as target color, its number as G₀With a weighted mean gray value of c₀(ii) a The pixel values above the threshold t are background colors, the number of which is denoted G₁With a weighted mean gray value of c₁Firstly, an optimal segmentation threshold is adaptively found for each wheat image, and the following formula can be obtained:

c＝b₀*c₀+b₁*c₁

t＝b₀*(c₀-c)²+b₁*(c₁-c)²

wherein, b₀The pixel value less than the threshold value t accounts for the proportion of the whole image; b₁The pixel value larger than the threshold value t accounts for the proportion of the whole image; c is the average gray value of the whole image; after iterationThe obtained maximum t value is the optimal segmentation threshold value;

after positioning the wormholes, setting the pixel value larger than the threshold value to be 255 and the rest part to be 0 by using a threshold value segmentation method, and quantifying the wormhole position characteristics to an area matrix (T) at the wormhole position T;

extracting and mapping the characteristics of the germinated grains to a frequency domain, and quantizing the characteristics into moment characteristics of different angles of the image as parameter extraction characteristics;

for the germinated kernels, first, spectrograms of the germinated and normal kernels were obtained and set to F₁And F₂；

Extracting characteristic parameters of entropy, energy, contrast and inverse difference moment of the spectrogram in directions of 0 degree, 45 degrees, 90 degrees and 135 degrees respectively to obtain the following formula:

wherein, Ent is the entropy of the image, Energy is the Energy value of the image, and Contrast is the Contrast of the image;

the wheat image is processed by the method, and then the wheat characteristics are extracted to identify the wheat.