AU765189B2 - Image data analysis of objects - Google Patents

Description

WO 01/04607 PCT/AU00/00830 1 IMAGE DATA ANALYSIS OF OBJECTS Field of the Invention This invention relates to image data analysis for objects such as meat carcases and meat cuts although the invention may also be applicable to other agricultural, mineral or manufactured objects.

Backeround of the Invention In the meat industry, specialist trained and skilled operators are employed, in abattoirs for example, in order to inspect each animal carcase and to provide estimates or gradings of various parameters, such as the predicted saleable meat yield of each carcase, Such predictions of meat yield and gradings are very important for fixing a fair value for the carcase and for determining uses to which the carcase and meat cuts will be destined.

Obviously it is very important for the meat industry generally including producers, processors and consumers that such operators are consistent both within a particular abattoir or processing facility and between different facilities at different place and different times.

There have been proposed and developed automated systems for carcase image capture and colour analysis for automating yield predictions or gradings, or at least for providing some objective replacement or supplement to human operators. For predicting the meat yield of a carcase, yield equations have been developed by statistical methodologies such as multiple regression analysis, such yield equations using the colour data to provide estimates of meat yield. However, the results of such automated analysis and yield predictions have not been of acceptable reliability or at least have been capable of significant improvement.

In the past, in order to predict the yield of meat carcase, i.e. the amount of saleable meat in the carcase, colour data captured by a colour video camera has been utilised in the SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 2 form of R, G, and B values (red, green and blue values) in yield equations derived from multiple field runs as described above. Particular care ought to be taken to ensure as far as possible that the R, G and B colour values are reliably and consistently measured both between different sites with different ambient conditions and using different cameras, and also throughout different periods of use, e.g. throughout a day, when lighting conditions can change. Our earlier patent application No. PCT/AU98/00135 (publication No. W098/39627) provides considerable detail concerning calibration procedures and systems for achieving the reliable and consistent colour measurements.

However we have found that even accurate and repeatable measurements in the form of R, G and B values when utilised in the relevant yield equations can provide predicted yields which are still susceptible of significantly improved accuracy or consistency.

Object of the Invention It is an object of the present invention to provide a method of analysing colour image data relating to a target object to derive or predict more accurately and consistently a property of the object of which the colour is an indicator.

Summary of the Invention According to the present invention there is provided a method of analysing colour image data relating to a target object to derive or predict a property of the object of which colour is an indicator, the method including the step of processing the colour data to derive light intensity independent measures of colour values, followed by the step of calculating the property of the object utilising the light intensity independent colour measures in a predictive equation in which the light intensity independent colour measures are variables and the property of the object is calculated from solving the predictive equation.

SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 3 Generally, in the field of meat quality grading, the light intensity independent measures of colour values are used in equations developed to predict a quantitative meat or carcase quality measure, e.g. "yield" in Australia, "conformation" or "fat score" in the EUROP grading system, or "yield grade" or "quality grade" in the USDA grading system.

For example, in the particular field of meat carcase yield prediction, the method includes the step of processing the colour data for a carcase to derive light intensity independent measures of colour values for the carcase, followed by the step of calculating the meat yield of the carcase utilising the light intensity independent colour measures in a yield predictive equation.

Description of the preferred embodiments It will be convenient to further describe the invention in relation to the particular field for which the invention has been developed, namely beef carcase yield prediction and grading, however it is to be understood that the principles, methods and systems can be adapted to other field of use.

Intensity Normalised Colour Space Considerable development of our beef carcase system for colour data capture and analysis has been towards achieving acceptable site-to-site consistency. It has been eventually established that the existing methods of lighting distribution compensation on a plane (particularly by calibration processes to minimise effects of lighting changes) did not adequately remove lighting variations in RGB space. To minimise effects of these variations, the present invention was developed involving use of an intensity normalised colour space.

That is, the intensity component was removed from the measurements leaving only colour.

Intensity Normalised Components SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 4 The intensity normalised class CRiGiI has been adapted from the prior CRGB class consisting of Red, Green and Blue values. The class consists of the member variables Ri; Gi; and I; where Ri is the intensity normalised red value, Gi is the intensity normalised green value, and I is the intensity. The calculation of these variables is described below. The intensity variable I is only used for reconstruction of the RGB tuple (set of values) and is not used in any yield equation calculations.

Calculation of RiGil from RGB The calculation of the intensity normalised values requires all red, green, and blue measurements of a RGB tuple. In addition, to ensure full intensity independence, a digitiser offset is preferably subtracted (since the offset associated with a digitiser for digitising measured RGB values in a colour data capture system is obviously not affected by light intensity variation). Through use of an assigned offset value and supply of the RGB values, the intensity normalised values are found as follows: (R -k) R R+G+B-3k (G-k) Gi R+G+B-3k (R+G+B-3k) 3 where k is the intensity normalised offset explained above.

Yield Equations In order to test the use of intensity normalised colour values in predicted yield or other grading measures, some test yield equations were developed from data gathered during a SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 WO 0104607PCT/AUOO100830 yield trial experiment. During this yield trial experiment, images for many beef carcases were captured at an operating abattoir and various measures obtained from these data were correlated with the saleable meat yield for each of the respective carcases. In this way the yield equations relating measured or calculated parameters of the carcases could be derived by multiple regression analysis (or other statistical analysis techniques) to best fit the data and optimise the fitting or prediction of the actual saleable meat yield.

At a general level of description, the predictive equation takes the form: Property x y.Ri z.Bi where x, y, and z are constants of positive or negative value derived by the statistical analysis techniques to best fit the data, and Ri is the intensity nornmalised red value, Gi is the intensity nornalised green value (or the intensity normalised blue value could be substituted for either the red or green value).

Two particular equations derived for the purpose of comparing the performance of a yield prediction equation using intensity normalised variables with a yield prediction equation using intensity based variables are as follows: Yield I 80.2 -35.4 x Ril 19.8 x Bi (1) Yield 2 77.48 -0.16 x R, 0.054 G, 0.094 x B, (2) where RA, is the intensity normalised red value of a predetermined "area V" of the carcase where a good predictive correlation between colour values and yield has been empirically determined, is the intensity normalised blue value of the same "area I, R, is the intensity based red value for the same "area I G, is the intensity based green value for the same "area and B, is the intensity based blue value for the same "area SUBSITIUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 6 These two equations were derived using the same data sets so as to provide equivalent equations for comparing predictive ability using one equation with intensity normalised colour values and the other using intensity based colour values.

To provide more variables and potentially greater predictive value, the predictive equation can takes the generalised form: Property a b.D c.Ri where a, b, and c are constants of positive or negative value derived by the statistical analysis techniques to best fit the data D is a dimensional parameter relating to the target object, and Ri is the intensity normalised red or green or blue value, the predictive equation optionally having further terms relating one or more further dimensional parameters relating to the target object and optionally having further intensity normalised red or green or blue value for the same or different sections of the area of the target object.

To test such equations having more variables, after testing and evaluation, a further two yield equations were derived from other data gathered at a different yield trial experiment conducted in an operating abattoir which included objective yield data from tissue sampling and laboratory fat analysis. These further equations are: Yield 3 72.31 0.0059 x d, 0.14 x f, 0.015 x s, 117.62 x G, 23.034 x R, 3 36.385 x Ri, (3) Yield 4 124.94 0.0039 x d, 0.42 x f, 0.026 x s, 0.26 x R, 0.13 x Gi 0.077 x B 3 (4) where d, is the distance from the tail to the hind leg bottom, when projected onto a longitudinal line through the carcass, d 2 is the distance from the brisket to the tail, SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 7 f, is the ratio wL, where w is the distance from the point where the hook suspending the beef carcass passes through the hind leg to the point at the end of the profile of the butt, when projected onto the longitudinal line, and L is the length of the carcass, f 2 is the ratio x/L, where x is the distance from the hook to the point of the armpit, when projected onto the longitudinal line, and L is the length of the carcass, s, is a measure of the degree of "plumpness" of the shape of the butt, e.g. derived by obtaining a measure of the extent of departure of the butt profile from the line from the point of the tail to the bottom of the hind leg, is the intensity normalised green value for a predetermined "area 2" of the carcass (different from "area 1 determined to have a predictive correlation to the yield, R- is the intensity normalised red value for a different "area 3" of the carcass, R is the intensity normalised red value for a different "area 4" of the carcass, and

B

3 is the intensity based red value for "area 3".

Some of these dimensional parameters are indicated on the accompanying drawing showing a beef carcase in side view as presented to the image capture camera.

It is to be appreciated that these yield equations were derived from particular sets of dimensional and colour data captured during particular yield trials conducted at operating abattoirs, including actual saleable meat yield data obtained using conventional grading techniques for each of the respective carcases. Hence the equations are illustrative only and different equations would result from the statistical analysis 'techniques used to derive these equations if applied to other sets of test data from carcases. For example only, very different equations would result from dimensional and colour data obtained for different species of beef cattle, different sexes of animals, different age groupings of cattle, different pasture or SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 8 feeding types and patterns for the cattle different types of grasses or pastures, grain fed versus grass fed, different climatic and seasonal conditions, dietary supplements and growth factor or hormone manipulation, etc.), different animal species (cattle, sheep, pigs, goats, etc), and possibly even mechanical processing variables (such as pelt or hide removal techniques which may affect the extent and location of fat left on the surface of the carcase). Hence these equations are illustrative only of the kind of equations that may be used in implementing the method of-. the present invention to calculate a property of an object utilising the light intensity independent colour measures in a predictive equation.

Also the derived equations will be different depending on the use of selected ones of the numerous variables including dimensional variables, ratios of dimensions, other measures such as the measure of the shape of the butt. The sizes and locations of the predetermined areas of the carcase where colour measurements are taken and used in the predictive equations will very substantially affect the final derived constants in the equations. These particular exemplified two pairs of predictive equations were derived using the same statistical methodology and using the same real data so as to thereby obtain comparative equations for testing the effect of using intensity normalised colour values.

Intra-site repeatability In order to test the stability of the beef carcase system over the duration of a further yield trial in an operating abattoir, a fake carcase having the size and shape of a real beef carcase and having its surfaces carefully coloured so as to closely match the fat and meat tissue colours of a real beef carcase was measured multiple times on a number of days during the period of the yield trials. Over a trial when the fake carcase was presented 37 times over a number of days, yield equations and exhibited only very small changes in the predicted yield minimum -0.062% and maximum +0.102% deviation from a median SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 9 predicted yield. On the other hand, yield equations and displayed a drift of minimum -0.39% and maximum +0.16% from the median.

Likewise the RMS of the changes of the individual yield predictors for the intensity normalised yield equations and had a maximum of 0.048, compared to the RMS of the changes in the individual intensity based predictor of equations and which had a maximum of about 0.3%.

These results show stability and repeatability of the system with the intensity normalised colour values showing substantially better intra-site repeatability.

In fact, during this intra-site test, one of the light bulbs of the carcase illumination system used during image capture failed in one carcase image capture and therefore significantly changed the carcase illumination conditions. The intensity normalised yield equations and showed only very small change in predicted yield and small change in the RMS of the changes for this trial with the failed light bulb, whereas the intensity dependent yield predictive equations and showed very large changes in predicted yield and RMS value for this trial with the failed light bulb. By chance, this demonstrated that the use of intensity normalised equations are robust to such changes in illumination conditions.

Inter-site repeatability To investigate the repeatability of the system between two sites, the fake carcase was presented to the imaging system at multiple times at a different meat processing plant. Using the four yield equations above, the predicted yields for the fake carcase were compared to the results from the other processing plant. These results demonstrated that the intensity normalised colour values showed excellent inter-site repeatability and, in particular, the results were substantially more consistent than with the predictive equations using intensity dependent colour values.

SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AUOO/00830 Lighting sensitivity To test the sensitivity of the system to lighting positioning during the capture of image data, the arrays of illuminating light sources used to illuminate the carcases were deliberately moved to different positions and images were captured at each of a number of different lighting positionings. Again the fake carcase was used to enable comparison of the results from the various yield equations. This experiment caused changes in yield predictions of for yield equations and (the intensity normalised colour values) compared to changes in predicted yield from equations and of Thus the use of the intensity normalised colour values shows robustness to lighting misalignment.

Animal type yield equations After the completion of numerous yield trials, it was determined that to achieve acceptable levels of yield prediction accuracy, multiple yield equations may need to be developed for different animal types, even when considering beef carcases alone. Equations based on both statistical methodologies and biological groupings were derived. Six categories were finally selected. These were: Bulls, Cows, Light Grasslike, Heavy Grasslike, Light Grainlike, and Heavy Grainlike. "Light" and "Heavy" refer to carcase weight, and "Grasslike" and "Grainlike" refer to tissue colour. The six equations were derived using statistical methodologies as outlined earlier. The equations have a generally similar appearance to the yield equations given earlier but have different variables (depending on which dimensional data, colour patch values, etc. show the best correlations and predictive ability for yield for each of the six carcase categories) and also different co-efficients.

SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 11 However, all six equations used intensity normalised colour values and showed good yield predictive ability and inter-site and intra-site repeatability and illumination insensitivity.

In use, each carcase was categorised into one of the six predetermined categories and the data captured was then used for the input to the respective equation for the relevant category to provide the yield prediction.

SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU0000830 12 Yield Equation Labels As mentioned, the BCS yield prediction accuracy relies upon application of the appropriate yield equation from one of the six categories mentioned above. The values for the Wy and CompWy for a particular beef carcase side will therefore have been derived from the yield equations applicable to its category.

Wy Predicted wholesale saleable meat yield CompWy Component predicted wholesale saleable meat yield for the BCS Combined Equation Weightings The BCS produces not only its prediction of saleable meat yield, but also a yield that forms part of a combinatorial equation. This component yield (CompWy) was added to a weighted CAS predicted yield representing the currently selected carcase type. "CAS" refers to a "Chiller Assessment System" (available from Viascan Quality Assessment, of Beenleigh, Queensland, Australia) which provides measures relating to meat yield after further analyses later in the processing operation in a chiller. This weighted addition must be applied off-line at the end of the processing operations in the abattoir. The formula that is implemented was was follows: Combined Wy BCS_CompWy k' x CAS_Wy where k' is defined according to which carcase type has been selected. The appropriate values are shown below in the table. Note that the CAS yield equations exist for only three carcase categories. These are: bull, cow, and table beef (where the "table beef' category includes all beef from the four subcategories of the BCS).

SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 13 BCS Yield k' CAS Yield Category Category Bull 0.4627 Bull Cow 0.8095 Cow Light grain 0.6057 Table beef Heavy grass 0.8136 Table beef Light grass 0.7751 Table beef Heavy grain 1 Table beef Table: CAS yield weightings by category Conclusions The use of colour normalised colour values in yield equations has been found to provide more accurate and therefore more reliable yield predictions of beef carcases than measured colour values which are influenced by light intensity at the time and place of the image capture even if considerable measures have been taken to calibrate the equipment to remove equipment, site and time induced variables and even if considerable measures have been taken to provide controlled lighting conditions at the image capture station.

The invention has been mostly described herein and illustrated in connection with predicting yield of a meat carcase, i.e. the amount of saleable meat, particularly a beef carcase. "Yield" is the primary measure used for carcase grading in Australia where the invention has been developed. However, in other countries or regions, there can be different parameters used to grade meat such as meat carcases.

For example, in Europe there is a scoring or grading system known as "EUROP" which involves determining one grading measure for the shape or "conformation" of a carcase (which categorises the degree of fatness or fullness of the carcase) and a "fat score" (which provides a score or grading for the overall fat coverage of the carcase). The present invention is equally applicable to the process of calculating the conformation and fat scores in SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 14 the EUROP system for a meat portion or carcase using light intensity independent colour measures in appropriate predictive equations. It will be appreciated that the capture of colour data for a carcase (together with other data such as dimensional data), can be used in an analogous manner to that described above for developing a yield predictive equation to develop equations to provide EUROP conformation and fat score measures. The use of light intensity independent measures of colour values in such conformation and fat score predictive equations will improve the intra-site and inter-site repeatability of the system and light orientation insensitivity as established for the yield predictive equations.

In the United States, there is a further meat grading system developed by the USDA.

This USDA grading system is based on analysis of the rib eye muscle colour and size and on the fat weight. The system involves the allocation of two grading measures known as the "yield grade" and the "quality grade". As with the EUROP grading system, the present invention using light intensity independent measures of colour values can be used in predictive equations for the "yield grade" and "quality grade" under the USDA grading system by developing such equations using multiple regression analysis techniques or other statistical methologies. The use of light intensity independent colour values in such equations for the USDA grading system will have the same advantages as described above for the Australian yield equations.

SUBSTITUTE SHEET (RULE 26) RO/AU

Claims (12)

1. A method of analysing colour image data relating to a target object to derive or predict a property of the object of which colour is an indicator, the method including the step of processing the colour data to derive light intensity independent measures of colour values, followed by the step of calculating the property of the object utilising the light intensity independent colour measures in a predictive equation in which the light intensity independent colour measures are variables and the property of the object is calculated from solving the predictive equation.

2 A method of analysing colour image data relating to a target object as claimed in claim wherein the colour image data comprising RGB colour values are obtained by digitising measured RGB values from a colour data capture system using a digitiser, the digitiser having a predetermined intensity normalised offset and wherein the light intensity independent measures of colour values are determined from the equations: (R k) Ri- R+G+B-3k (G-k) R+G+B-3k (R+G+B-3k) 3 SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 16 where Ri is the intensity normalised red value, Gi is the intensity normalised green value, and I is the intensity, the intensity variable I being only used for reconstruction of the RGB colour values.

3. A method of analysing colour image data as claimed in claim 1 or 2 wherein the predictive equation is developed from data gathered during a data gathering experiment using images captured for real target objects, the method comprising correlating the light intensity independent colour measures obtained from these data with the actual measured property of each of the real target objects to derive the predictive equation by statistical analysis techniques to best fit the data and optimise the prediction of the actual measured property from the light intensity independent colour measures.

4 A method of analysing colour image data as claimed in claim 3 wherein the predictive equation takes the form: Property x y.Ri z.Bi where x, y, and z are constants of positive or negative value derived by the statistical analysis techniques to best fit the data and Ri is the intensity normalised red or blue value, Gi is the intensity normalised green or blue value.

A method of analysing colour image data as claimed in claim 3 wherein the predictive equation takes the form: Property a b.D c.Ri where a, b, and c are constants of positive or negative value derived by the statistical analysis techniques to best fit the data, D is a dimensional parameter relating to the target object, and Ri is the intensity normalised red or green or blue value, the predictive equation optionally having further terms relating one or more further dimensional parameters relating to the target SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 17 object and further intensity normalised red or green or blue value for the same or different sections of the area of the target object.

6. A method of analysing colour image data as claimed in any one of the preceding claims wherein the object is a meat object, the property of the meat object being a quantative meat or carcase quality measure, the method including the steps of capturing and processing colour data for the meat object to derive light intensity independent measures of colour values, followed by the step of calculating the quantative meat or carcase quality measure for the meat object utilising the light intensity independent colour measures in a predictive equation in which the light intensity independent colour measures are variables and the quantative meat or carcase quality measure of the meat object is calculated from solving the predictive equation.

7. A method of analysing colour image data as claimed in claim 6 wherein the quantative meat or carcase quality measure is a measure selected from the set consisting of: the "yield" of a carcase in a standard carcase grading system, the "conformation" of a carcase in a standard carcase grading system, the "fat score" of a carcase in a standard carcase grading system, the "yield grade" of meat from a carcase in a standard meat grading system, and the "quality grade" of meat from a carcase in a standard meat grading system.

8. A method of analysing colour image data as claimed in claim 7 wherein the quantative meat or carcase quality measure comprises the "yield" of a carcase as defined in the standard Australian carcase grading system.

9. A method of analysing colour image data as claimed in claim 7 wherein the quantative meat or carcase quality measure comprises the "conformation" of a carcase in the EUROP standard carcase grading system.

SUBSTITUTE SHEET (RULE 26) RO/AU WO 01/04607 PCT/AU00/00830 18 A method of analysing colour image data as claimed in claim 7 wherein the quantative meat or carcase quality measure comprises the "fat score" of a carcase in the EUROP standard carcase grading system.

11. A method of analysing colour image data as claimed in claim 7 wherein the quantative meat or carcase quality measure comprises the "yield grade" of a meat object in the USDA standard meat grading system.

12. A method of analysing colour image data as claimed in claim 7 wherein the quantative meat or carcase quality measure comprises the "quality grade" of a meat object in the USDA standard meat grading system. SUBSTITUTE SHEET (RULE 26) RO/AU