SE470465B - Method and apparatus for automatic assessment of grain cores and other granular products - Google Patents

Description

'iZi co .iïx (j \ 10 15 20 25 30 35 CT] 2 There are examples in the scientific literature that attempts have been made to assess cereals with the help of computerized image analysis. In an article by Sapirstein et al in Cereal Science no. 6, 1987, page 3 describes, for example, an attempt to distinguish between wheat, rye, barley and oat kernels with the help of different contour parameters.By analyzing a statistically calculated combination of length / width, width, moment and circumference, circuit length, an image analysis program could identify cores with an accuracy of more than 97%.

An article by Zayas et al in Cereal Chemistry 66 (3), 1989, page 233 describes an image analysis system for determining "non-wheat components" in samples of wheat. The shape of the wheat kernels is described with 10 geometric parameters and a measure of the color of the wheat kernels expressed in grayscale.

However, the above-mentioned systems are not commercially useful because they are experimental systems and are based on the cores being presented manually one by one for the image analysis systems.

The patent literature also contains examples of systems for automatic assessment of cereal kernels and other objects.

GB 2 012 948 shows a method for determining the size distribution for samples of, among other things, cereal kernels. According to this method, the cores are caused to fall between a screen, which is illuminated by a stroboscope, and a video camera, with which pictures are taken of the cores. The video images are digitized and the cores are identified in the images. Based on the size of each image of a core, the size distribution of the cores in the sample is determined. With this method, however, the cores cannot be classified.

WO 91/17525 describes a method for automatically classifying an object into predetermined classes. According to this method, a camcorder takes time domain images of objects fed one at a time on a conveyor belt past the camera.

The time domain images are transformed by Fourier analysis into frequency domain signals, which form inputs to a 10 15 20 25 30 35 __T s. S ä.

The CW (H 3 neural network, which performs the classification itself) is obtained. With this method, not enough objects can be analyzed per unit of time for the method to be commercially useful for grain classification. The object of the present invention is to provide a method and a device for automatic assessment of granular products handled in bulk form, in particular cereal kernels, which method and device can replace human inspection and assessment.In order for such a method and device to be commercially applicable, a sample must be able to be analyzed more or less the same time as it takes today to analyze it manually.In particular, it means that a sample of about 1500 grain cores must be able to be classified in about 5 minutes.In addition, the accuracy of the classification must be high.The percentage of weight distribution of the various constituents in For example, a wheat sample should be able to be determined with an accuracy of about 0.2% of the whole weight of the sample. target samples may contain stones and other foreign objects is an additional requirement that such objects must be identifiable in the assessment. In addition, the device and the method should be able to determine the size distribution and color distribution of the sample. The object is achieved with a method and a device which have the features stated in the claims.

The method and device according to the invention have the advantage that a sample of cereal kernels can be analyzed as quickly as if the analysis was done manually. This is made possible by presenting a plurality of cores at a time for a device which produces digital images of the cores, the cores being oriented in one direction during the presentation. By presenting the cores in this way, they can be quickly and securely identified in the digital images.

The nuclei are classified using a neural network, the input signals of which are based on pixel values for a number of pixels representing the nucleus. It has been found that the use of pixel values as assessed CD -F- Cïs 10 15 20 25 30 35 CT! The basis for the classification provides good accuracy.

It should be pointed out that by pixel value is meant here a value used to represent the pixel; eg the intensity of monochrome images; red, green, and blue intensities at RGB representation in color images; color tone, whiteness and brightness at HSI representation in color images, etc.

To increase the speed of the device, it is advantageous to form the input signals to the neural network by summing the pixel values for a plurality of pixels, so that the information content of the pixels representing a nucleus is compressed.

When using color images, a component-by-sum sum for each pixel component in a plurality of pixels has been shown to provide good accuracy.

It has also proved to be an advantage to switch from RGB representation to HSI representation, since the latter representation provides higher accuracy when classifying grain kernels.

Through empirical experiments, it has been shown that one can determine a relationship between the size of the image of a nucleus and the weight of the nucleus. This is used to determine the weight of the cores.

To avoid two or more cores lying next to each other being incorrectly perceived as a single core, the extent of each contiguous area of pixels representing a core perpendicular to the longitudinal axis of the area is determined and it is examined whether the extent has one (or more) minimum of other than at the ends of the area. If this is the case, the image is judged to contain two (or more) cores and is divided by the minimum (minimum).

This simple and fast separation of cores is made possible by the cores being oriented in one direction.

Embodiments of the device according to the invention are specified in the subclaims.

The present invention will now be described by way of example with reference to the accompanying drawing, in which the single figure shows an embodiment of a device according to the invention, the feed device being shown in longitudinal section and the image processing device being shown in block diagram form.

The invention consists, as shown in the figure, essentially of a feed device 1, a video camera 40 and an image processing device 2. The feed device 1 comprises a first belt conveyor 3, which is arranged in a housing 4 and has a first wheel 5, which is driven by a not shown motor, a second wheel 6 and an endless belt 7 running around the wheels 5, 6. The belt 7 has grooves 8, in which the grain kernels are portioned out. Alternatively, the strap may have differently shaped recesses.

In the housing 4 there is furthermore a magazine 9, which tapers in the direction of the belt 7 and in which a sample of grain kernels is filled. The magazine 9 comprises two plates 10, 11, which are inclined in relation to each other. The lower end of one plate 10 is spaced from the belt 7 and to this end is attached a wiping means 12, which is arranged to lower the grain cores into the grooves 8.

A second belt conveyor 15 is arranged longitudinally. and height displaced relative to the first belt conveyor 3. This second belt conveyor 15 comprises a first wheel 16, which is driven by a motor (not shown), a second wheel 17 and an endless belt 18, which runs over the first and the second wheel 16, 17. The belt 18 has grooves 14 in which the cores are transported. The grooves 14 in the second conveyor are denser than those in the first conveyor and their width is adapted to cores in a certain size range, so that the cores orient themselves in the longitudinal direction of the notches. The color of the ribbon is chosen to create a strong contrast with the background. For analysis of grain kernels, for example, purple can be used to advantage. Between the first wheel 16 and the second wheel 17 there is also a third wheel 19, the function of which is explained below. The first wheel of the second belt conveyor 15 -Fzs -J CD 10 15 20 25 30 35 CF] 6 16 is arranged under the second wheel 6 of the first belt conveyor 3, so that grain cores can fall from the first conveyor 3 onto the second conveyor 15. Two plates 20, 21 are arranged between the first belt conveyor 3 and the second belt conveyor 15. When the cores fall from the first conveyor, they bounce first against the plate 20 and then against the plate 21, whereby a spreading of the cores is effected. At the sides of the second belt conveyor there are at its first wheels 16 boundary pieces 22, which ensure that the grains are initially at a certain distance from the edges of the belt 18. On the front end of the boundary pieces 22 in the direction of the belt there is a curtain 23 which is arranged to lower the cores into the notches on the endless belt 18 and to ensure that the cores lie in a layer and do not overlap each other.

Between the first wheel 16 and the third wheel 19 and between the upper and lower part of the belt 18 a vibrating device 25 is arranged. This device comprises a shaft 26, to which one end of a plate 27 is attached. Its other end is arranged between a roller 28, which is driven by a motor (not shown), and the underside of the belt 18. On the short side of the roller 28, three washers 29 are mounted with winches by means of screws. As the motor rotates the roller 28, the plate 27 will strike the belt at a fixed frequency and generate a vibration in the belt 18. The amplitude of the vibration is determined by the position of the roller 28 and the play of the washers. Preferably, the amplitude should be the same regardless of the stiffness of the belt.

At the third wheel 19 a gear detection unit 31 is arranged. This is mounted on one side of the periphery of the third wheel 19 and comprises a light emitter in the form of an LED 32 and a light receiver in the form of a photocell 33. The tooth detection unit 31 is connected (not shown) to a computer 42. When the third the wheel 19 rotates, the gear detecting unit 31 emits a pulse-shaped signal to the computer 42. The third wheel 19 further serves to dampen vibrations in the belt 18 in the area between the third wheel 19 and the second wheel 17. '-f fl 10 15 20 25 30 35. Above the endless belt 18 in its area near the second wheel 17, a video camera 40 is arranged so that pictures of the belt 18 in the vicinity of the second wheel 17 can be taken. To improve the illumination of the tape, an annular lamp 41 is arranged between the camera and the tape.

The camera 40 is connected to the image processing device 2, the structure and function of which will be described in more detail below.

In the following, the function of the feed device 1 will be described. A sample of grain kernels is poured onto the first belt conveyor 3 through the magazine 9.

The cores then lie in a pile on the belt, but when this moves they will, due to the belt sloping upwards and through the care of the wiping member 12, be spread out in portions in the grooves 8 of the belt. When the cores reach the second wheel 6, they fall down, bounce against the plates 20 and 21 and spread on the second belt 18, preventing them from settling on the edges of this belt by the boundary pieces 22. As the second belt 18 vibrates, the cores will they are transported forward to move laterally in the grooves towards the edges of the belt. Cores lying on the axes between the grooves will fall into the grooves. Therefore, when the cores reach the area below the camcorder 40, they will be separated in the longitudinal direction of the tape, be oriented in substantially the same direction, and lie in substantially only one layer on the tape. The cores will thus overlap each other to a very small extent. On the other hand, the cores can lie next to each other in the grooves in their longitudinal direction.

Each thread that the computer 42 has counted to a predetermined number of teeth receives an interrupt signal, whereby the computer 42 stops all drive motors. The first and second bands then stop and the vibration stops. After a short wait, the computer 42 gives a signal to the camcorder 40 which takes a picture of the cores on the tape 18. Then the motors are restarted and the feeding of cores continues as above until an interrupt signal is obtained again. The reason why the system waits after the belt conveyor has been stopped is that all movements of the cores must be damped out so that they are stationary. As mentioned, the third wheel 19 helps to reduce the amplitude of the vibrations in the area under the camera 40 so that the waiting time can be kept short. The predetermined number of teeth after which the interrupt signal is given is chosen so that the camcorder will take pictures of the tape, which images cover the tape without gaps, but without overlaps. In other words, each core that passes the camcorder will be found in exactly one image and each image will contain a plurality of cores.

Alternatively, the tape can be fed continuously and the lamp 41 replaced with a stroboscope, which together with the camera 40 is controlled so that pictures are taken of the tape without gaps and without overlaps.

The image processing device 2 basically comprises a computer 42, which is connected to the video camera 40, and a terminal 43, on the screen of which the result of the analysis is presented. The computer 42 contains programs for classifying and other assessment of the grain kernels based on the images obtained by the video camera 40. These programs include a conversion of the video signals from the camera 40 into appropriate inputs to a neural network program that performs the classification itself.

When the camcorder 40 has taken a picture of the tape, this picture is read into the computer and digitized by means of a known so-called frame grabber. The resulting digitized image consists, for example, of 512 x 512 pixels. The pixels are represented by RGB representation, ie by a value for the intensity of red color, a value for the intensity of green color and a value for the intensity of blue color. Alternatively, a grass scale can be used.

In the next step, the program locates the cores of the digitized image. In this case, a threshold value for the color in each pixel is used. To simplify the processing in this step, it is advantageous to switch from RGB representation to HSI representation (Hue, Saturation and Intensity = color tone, whiteness and brightness). When the value of a 10 15 20 25 30 35 H »see CD fw Cm (fl 9 pixel exceeds the threshold value, the pixel is assumed to represent a core, while when the value is below the threshold value, the pixel is assumed to represent background. The program examines the image point by point, line When it encounters a pixel representing a core, it examines all adjacent pixels.For the pixels of the adjacent pixels that are considered to represent a core, the operation is repeated and this continues until all the pixels associated with the first pixel have been identified. the longitudinal axis of the contiguous pixels as representing a core.

If the direction of the longitudinal axis deviates by more than a predetermined value from the y-axis of the image, the continuous core area is rotated so that its longitudinal axis becomes parallel to the y-axis of the image.

When the image of the cores on the belt is taken, it may happen that two or more cores lie next to each other in a cut on the belt or even overlap each other to a certain extent. The coherent core area identified in the image can thus represent more than one core.

To check if this is the case, the number of pixels in the x-direction that represent a core is summed for each y-value in the continuous core area. The program thus makes a histogram of the number of core pixels in the x-direction.

Then an envelope curve is determined for the histogram and it is examined whether there is any minimum between the endpoints of the envelope curve in the y-direction. A sufficiently strong minimum indicates that the contiguous core pixel area actually corresponds to two cores. If so, the program makes an incision parallel to the x-axis at the minimum of the envelope curve. Thereafter, each part of the contiguous core pixel area is stored as an image of a core. If there are several minimums, an average is added to each minimum. If a separation of a core pixel area has been performed, the longitudinal axis of each core is determined and the core is rotated if the deviation from the y-axis of the image is greater than the predetermined value. The reason for this \ a LTD 10 15 20 25 30 35 (, "\ CT! 10 is that when the determination of the longitudinal axis is made before the separation, the image may not be rotated or rotated incorrectly because the parts corresponding to each core are both inclined in relation to the y-axis of the image in such a way that the common longitudinal axis of the core pixel area corresponds to the y-axis.After the separation, each nucleus then tilts relative to the y-axis, which is a disadvantage in the classification.

In order to avoid that the image processing device picks up stones and other debris that may be present among the cores as cores, an assessment is made of the size of the continuous pixel area. If this is not within a certain range, it is considered to represent a foreign object and is registered as such.

In the next step, the RGB values of the pixels are converted to HSI values. This conversion is not necessary, but it has been shown that the classification of cereal kernels becomes more correct if HSI representation is used instead of RGB representation.

In the following steps, the H-values separately, the I-values separately and the S-values separately along rows and columns in the image of a core are summed. For each y-coordinate, the values of the H-component for all x-coordinates are first summed.

Then a corresponding summation is made for the I value and the S value. Then for each x-coordinate the H-value for all the y-coordinates is summed, after which the summing is repeated for the S- and I-values. The program thus creates a histogram in the x-direction and one in the y-direction for each pixel component. You then receive a large number of sums. These sums are standardized, whereby the mean value and the standard deviation of the corresponding sums for previously classified cores are used in such a way that if the value of the current sum is equal to the mean value for previously classified kernels, its standardized value is set to zero and if the current sum value deviates more than 12.5 standard deviations from the mean value, its value is set to il. Sums in between are normalized in proportion to the mean value to values between -1 and +1. 10 15 20 25 30 35 l 13 * x f? Q fi 11 The standardized sums form input signals to a neural network. A neural network is a program consisting of a number of inodes, in this case one for each sum, and a number of nodes, which in this case represent each of the possible classes into which the nuclei can be classified.

There are hidden nodes between the inodes and the outputs. By feeding inputs that represent known nuclei to the neural network and telling the class in which the nucleus is to be classified, the neural network can be "trained" to classify nuclei correctly. Once the neuronal network has learned to classify the various interesting nuclei, it can - used to classify unknown cores.The hidden nodes consist of sigmoid functions, which enables the adaptation of input data to a largely arbitrary (linear / non-linear) function.In cases where the classes are linearly dependent on the inodes the network is "trained" to make a linear discriminant adaptation, so the neural network method contains linear discriminant adaptation as a special case.

Each node is represented by a value between and 0 and 1. The sum of the values of the nodes is equal to 1. When classifying, a core is judged to belong to the class whose corresponding node has the highest value. However, it is also possible to favor a certain type of grain. For this purpose, samples are taken before classification, the type of grain that is dominant is determined and this is stated to the neural network. If the largest node value is less than a predetermined value and the node that is favored has the second largest value, then the core is not classified in the class whose node has the greatest value but in the class whose node has the second largest value.

The results of the classification are presented on the terminal's 43 screen, for example in the form of a histogram with a bar for each cereal, one for flying oats, one for burnt kernels and one for damaged kernels.

The result can be presented in% by weight of the sample weight.

Namely, it has been found possible to determine the weight of the core by means of the size of its image, since ~ q cs .Fa.

Ch U'l 10 15 20 25 30 35 12 there is a relationship, which can be determined empirically, between these parameters. The assessment thus counts the number of pixels that represent the core in question. From this number, the size and weight of each core can be determined and thereby the entire weight and size distribution of the sample. C The color distribution of the sample can also be determined from the input signals to the neural network.

The table below shows an example of ten analyzes of a 50 g grain sample analyzed with a device according to the invention. The sample consisted of: 5.00% rye; 5.00% oats; 5.00% grain; 5.00% roasted wheat kernels; 0.00% flying oats; 5.00% damaged wheat kernels and 75.00% wheat. x is the mean and s (x) is the standard deviation. All values are% by weight of the sample weight.

Wheat Rye Oats Barley Roasted Flygh. Injured 74.42 4.97 4.70 5.02 5.72 0.00 5.17 74.37 5.11 4.61 5.33 5.32 '0.90 5.21 74.84 4.96 4.91 4.87 5.33 0.05 5.04 75.42 5.31 4.78 4.85 4.65 0.20 4.78 74.94 5.13 4.77 4.73 4, 92 0.09 5.42 74.63 5.08 4.92 5.00 5.01 0.00 5.35 74.79 5.27 4.63 5.23 4.98 0.00 5.10 74 , 36 5.54 4.80 4.95 5.95 0.03 4.92 74.15 5.35 4.60 5.38 "5.58 0.00 4.93 74.93 5.69 4, 50 4.86 4.85 0.02 5.15 x 74.68 5.24 4.72 5.03 5.18 0.04 5.11 s (x) 00.36 0.23 0.13 0, 27 0.37 0.06 0.19 The values above are expected to be improved by basing the relationship for conversion from size to weight on a larger number of cores.

Claims (10)

10 15 20 25 30 35 .fx »a c: .flm - o \ m 13 PATENTKRAV Methods of automatically assessing grain kernels or similar granular products handled in bulk, characterized by the kernels spreading and orienting in a predetermined direction; digital images of the cores are created, each image containing a plurality of cores, but each core appearing in only one image; that a set of input signals to a neural network is created for each nucleus by means of pixel values for a plurality of the pixels representing the nucleus; and that the neural network determines to which of a plurality of predetermined classes each nucleus belongs. 2. A method according to claim 1, characterized in that the creation of input signals to the neural network comprises summing the pixel values for a plurality of the pixels representing the nucleus. 3. A method according to claim 2, characterized in that the summing of the pixel values is done componentically for each pixel component in a plurality of pixels representing the core. A method according to any one of claims 1-3, characterized in that the pixels in the digital image are represented by the components red-intensity, green-intensity and blue-intensity (RGB representation) and that these pixel components are converted into a hue component , a brightness component and a brightness component feel - (HSI representation). 5. A method according to any one of the preceding claims, characterized in that the cores in the digital images can be - identified by locating contiguous areas of pixels with an intensity or color exceeding a predetermined value, determining the extent of each such contiguous area perpendicular to the longitudinal axis of the area and at each minimum to the extent which has at least a predetermined size and is Ås a (__) \ on 14 spaced from the ends of the area in the longitudinal direction the area is divided, each part thus obtained being treated as representing a core. 6. A method according to any one of claims 1-5, characterized in that the weight of each core is determined on the basis of the size of the area of pixels representing the core. Device for the automatic assessment of cereal kernels or similar granular products handled in bulk, characterized by a device (40) for producing digital images of the kernels in such a way that each kernel is present in one and only one image, a display device (14-19, 25-28), which is arranged to present a plurality of cores at a time substantially oriented in a direction in the field of view of the device (40), and a neural network for classifying the cores. Device according to claim 7, characterized in that the presentation device (14-19, 25-28) comprises a belt conveyor (15), the belts of which have depressions (14), the shape of which is adapted to the cores and which are oriented in a common direction. Device according to claim 8, characterized in that the presentation device (14-19, 25-28) comprises vibrating means (25-28), which are arranged to vibrate the conveyor (14) for orienting the cores on the conveyor (14). . Device according to claim 9, characterized in that the vibrating means (25-28) comprise a plate (27), one end of which is arranged between the belt (14) and a vibrating element, which comprises a roller (28) and a a plurality of washers (29) mounted with winches on the roller, so that the washers (29) strike the plate (27) when the roller (28) is rotated.