WO1991004803A1

WO1991004803A1 - Classifying and sorting of objects

Info

Publication number: WO1991004803A1
Application number: PCT/AU1990/000464
Authority: WO
Inventors: Ian Robert Madden; Charles Edward Esson; Peter Gary Brown
Original assignee: Colour Vision Systems Limited
Priority date: 1989-09-27
Filing date: 1990-09-28
Publication date: 1991-04-18

Abstract

Apparatus for weighing, sizing and defect sorting of fruit comprising one or more singulators (11) to form the fruit into one or more rows, a conveyor that is arranged to pass the rows of fruit underneath a charged couple device array camera (19), means (98, 99) to rotate each piece of fruit (100) as it passes underneath the camera (19), an image processing means for processing a captured image, the image processing means comprises a master processor (102) which passes signals to a vision processor (103) and onto at least eight object processors (104-111) which divide the captured image into image sections with each image section containing a representation of only one of the pieces of fruit. The object processors (104-111) simultaneously process each image section and communicate information on the processed sections to control means. The control means includes means to classify the objects according to predetermined criteria using the analysed output. The predetermined criteria can relate to size, weight and especially colour. The apparatus includes ejection means (50) spaced along the length of the conveyor controlled by the control means to eject the fruit to outfeed conveyors (15) at particular locations corresponding to the predetermined criteria.

Description

Title

CLASSIFYING AND SORTING OF OBJECTS Field of Invention

This invention relates to the classifying and sorting of objects and includes a sizing, colouring, and defect sorting conveyor system for use with a wide variety of both primary produce and packaged and unpackaged foodstuffs. In particular, the invention relates to a conveyor system that is used for the colour and blemish sorting and dimensional sizing of various fresh fruits. Prior Art

Over the years there have been many proposals associated with the sizing and weighing of foodstuffs, especially fresh fruit. There are a number of sophisticated conveyor systems that weigh and size fruit that is passed along the conveyor system to automatically eject the fruit at predetermined locations so that the fruit can be sorted and collected in accordance with size and weight. There have also been proposals to incorporate photographic means to take pictures of the fruit as they pass along the conveyor in an attempt to colour and blemish sort the fruit. However, these attempts to colour and blemish sort fruit have been only partially successful and it is considered that there is a need for a far more sophisticated and accurate colour and blemish sorter that can be used not only with fruit, but many other types of primary produce and both packaged and unpackaged foodstuffs.

It is this need that has brought about the present invention. Summary of the Invention

According to the present invention there is provided apparatus for classifying and sorting objects comprising: a singulator to form the objects into a least one row; a conveyor that is arranged to pass at least one row of the objects underneath a camera, the camera being capable of capturing at least one image of the objects; means to rotate each object whilst passing underneath the camera; an image processing means for processing of the captured image, the image processing means including means for dividing the captured image into image sections, each image section containing a representation of only one of the objects, and the image processing means in use performing further processing on each of the image sections simultaneously; and a control means for analysing output from the image processing means, classifying the objects according to predetermined criteria using the analysed output, and controlling ejection means to reject the objects at particular locations on the conveyor corresponding to the predetermined criteria. According to the present invention there is further provided a method of classifying and sorting objects comprising the steps of: singulating the objects into at least one row; passing at least one row of the objects on a conveyor past a camera; rotating each object whilst passing underneath the camera; capturing at least one image of each object; digitising the captured image; dividing the captured image into a plurality of image sections, each image section containing a representation of only one object; analysing the plurality of image sections simultaneously to determine information on the colour, size and/or shape of each object; classifying the objects by comparing the determined information of each object with predetermined criteria; and selectively ejecting the objects from the conveyor at stations corresponding to the predetermined criteria. Description of the Drawings

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which:

Figure 1 is a plan view of apparatus for size and blemish sorting fruit.

Figure 2 is a side elevational view of the apparatus in Figure 1,

Figure 3 is a cross-sectional view taken along the lines 3-3 of Figure 1 illustrating a photographic section,

Figure 4 is an illustration of a view that would be seen by a camera in the photographic section.

Figure 5 is a cross-sectional view taken along the lines 5-5 of Figure 1,

Figure 6 is a cross-sectional view taken along the lines 6-6 of Figure 5, Figure 7 is a perspective view of a roller shaft chain clip forming part of the conveyor,

Figure 8 is a perspective view of an ejector lever chain clip and ejector lever forming part of the conveyor, Figure 9 is a sectional view taken along the lines 9-9 of Figure 1 illustrating an ejection station,

Figure 10 is a cross-sectional view taken along the lines 10-10 of Figure 9,

Figure 11 is a cross-sectional view similar to Figure 1 but illustrating a second embodiment of the invention,

Figure 12 is a block diagram illustrating the computer of the classification system connected to a single camera,

Figure 13 illustrates the functional equivalent of Figure 12 and the camera/conveyor synchronization process,

Figure 14 shows a functional diagram of a two camera system,

Figure 15 illustrates the human eye's sensitivity to colour,

Figure 16 illustrates the RGB representation of a coloured point.

Figure 17 shows slices of image sections used by the object processors,

Figure 18 is a graphical representation of the two-dimensional classification map,

Figure 19 shows a classification criteria boundaries on the classification map,

Figure 20 illustrates the object volume estimation technique, and

Figure 21 illustrates the object diameter estimation technique. Description of the Embodiments

As shown in Figures 1 and 2 a conveyor system for fresh fruit comprises essentially a feed conveyor 10 that feeds the fruit to a singulator 11 that positions the fruit individually in two rows, each piece of fruit being supported centrally between a pair of rollers. The two parallel rows of separated fruit pass under a photographic section 12 to an ejection section 13 which includes a plurality of spaced ejector positions on either side of the assembly that have the effect of causing particular fruit to be ejected onto a plurality of outfeed conveyors 15 that extend substantially transversely to the direction of movement of the main conveyor. The conveyor system is supported about a metallic framework 29 to be freestanding as shown in Figure 1.

The conveyor system operates so that the fruit are fed from the feed conveyor 10 onto the singulator 11 which causes the fruit to pass in two separate rows through the photographic section 12. Whilst in the photographic section 12, the fruit are rotated and various photographs are taken by a charged couple device (CCD) array camera 19 to provide a full colour video signal corresponding to each photograph. A computer comprising a series of microprocessors makes quality and quantity decisions about the fruit that are photographed by the camera. A master processor unit comprising a system of master processor and camera processor communicates with a vision processor and a plurality of object processors to classify the fruit according to predetermined sizing, colour and defect criteria. As the fruit pass through the photographic section 12, and into the ejection section 13, the computer periodically signals ejector assemblies to eject certain fruit at corresponding ejector positions on the conveyor in accordance with the classification criteria used. These fruit are fed into outfeed conveyors 15 which then transport the fruit to suitable storage means. In the first part of this description, a detailed explanation is given of the mechanical componentary that makes up the conveyor system and in the second part of this description, there is reference to the theory and technology that underlies the fruit classification means that utilises the camera signal to achieve the end result of sorting and ejecting certain fruit from the conveyor. Mechanical Details

As shown in Figures 1 and 2, the conveyor includes an infeed conveyor 10 which is of conventional form that supplies the fruit to the conveyor system. The infeed conveyor is arranged to feed fruit to the singulator 11 which comprises a horizontally positioned conveying surface 110 having inclined singulator conveyors 111 and 112 positioned on either side. The inclined conveyors 111 and 112 operate at half the speed of the horizontal conveyor 110 and have a series of spaced plates extending across the conveying surfaces. The inclined conveyors 111 and 112 cooperate with the horizontal conveyor 110 to, in effect juggle the fruit to ensure that they are formed into a single row so that they can feed in single file to the main conveyor system. Although not shown in the drawings, the inclined singulator conveyors 111 and 112 are arranged to extend past the end of the horizontal singulator conveyor

110 alongside the start of the main conveyor. In this manner, in the event that more than one piece of fruit attempts to sit in the gaps between roller assemblies that form part of the main conveyor, the fruit is joggled side¬ ways against the slowly moving singulator conveyor belts

111 and 112 and is held there until a space appears between two rollers thereby ensuring that complete singulation takes place. The inclination of the singulator conveyors 111 and 112 is approximately 30 degrees to the horizontal.

The main conveyor system which includes the singulator 11, photographic section 12 and ejection section 13 comprises a pair of spaced apart continuous chains 20 each driven by a pair of sprockets 21, 22 at either end. It is understood that the drive to the sprockets 21, 22 is undertaken by conventional means. The pair of continuous chains are shown in cross section in Figure 5 where it can be seen that each chain 20 runs on an elongate plastics runner 25 that is secured to extend longitudinally of an extruded aluminium section 26. The aluminium section has upstanding arm portions 27, 28 that fit and locate the runners in two spaced apart parallel horizontal arrays. The extruded aluminium section 26 extends along the underside of the upper run of the conveyor. The chains 20 are used to drive a plurality of parallel extending spaced apart roller assemblies 30 each of which extends transversely of the chains and comprises a transverse drive shaft 31 that coaxially supports a plastics roller 32 with a concavely tapered exterior surface 33. The rollers 32 are mounted on the drive shaft 31 so that rotation of the drive shaft 31 can impart rotation to the rollers. The drive shaft 31 is attached to the chains 20 through a plurality of roller shaft chain clips 40, shown in cross section in Figure 5 and perεpectively in Figure 7. Each roller shaft chain clip 40 comprises a plastics moulding with a central rectangular shaped box section 41 having a pair of coaxially aligned apertures 42 through which the drive shaft 31 freely extends. The rectangular box section 41 terminates in a pair of downwardly extending legs 44 which terminate in inwardly facing clip portions 45 which are arranged to clip under the link 29 of the chain 20 as shown in Figures 5 and 7. The chain clips 40 allow the drive shafts 31 to be clipped across the chains 20 so that movement of the chains by the spigot drive causes the roller assemblies 30 to move along the conveyor. Each roller 32 with its concave outer surface 33 is arranged so that each fruit 100 on entering the conveyor from the feed conveyor 11 is able to roll into a freestanding position where it is wedged between adjacent rollers, see Figures 2 and 3. The spacing of the rollers and their concave outer surfaces allow fruit of various sizes to be supported between adjacent rollers. As the fruit passes up the singulation section 11 of the conveyor, the rollers 32 shuffle the fruit into position so that they enter the photographic section 12 supported in equilibrium between adjacent rollers 32.

As shown in Figure 3, the photographic section 12 comprises an enclosure 95 that is positioned with its open base 61 closely above the conveyor, the enclosure 95 has its interior side walls 62 with a suitable reflective material. A pair of electric lights 63 are positioned adjacent the upper end of the rectangular enclosure with the light pointing towards the corners of the enclosure 95 as shown in Figure 2. The CCD video camera 19 is positioned centrally of the enclosure with the lens 64 projecting downwardly to point at the two rows of fruit that are passing through the photographic section. In Figure 4, there is an illustration of what the camera sees as it takes a picture of the two rows of fruit as they pass through the enclosure. The picture comprises eight rectangles that overlap at adjacent edges so that some of the rectangles embrace a single piece of fruit and the edges of the adjacent fruit. The images in the rectangles are transferred to eight object processors for interpretation as described later in the specification.

As shown in Figure 5 a circular wheel 98 is attached to the mid point of each roller support shaft 31. When the support shafts pass into the photographic section 12, the wheels 98 are arranged to ride against an abutment 99 that extends upwardly from the aluminium section 26. When the wheel 98 engages the abutment 99, the longitudinal motion of the conveyor causes the wheel 98 to rotate to drive the shaft 31 to in turn drive the rollers 33. This has the effect of causing the fruit located between adjacent rollers, to rotate through 360 degrees as the fruit passes through the photographic section. Rotation of the fruit ensures that all sides of the fruit are exposed to the camera whilst four photographic shots are taken.

Once the fruit has passed photographic section 12, the fruit reaches the ejection section 13 which comprises a plurality of longitudinally spaced ejector assemblies 50 shown in Figures 1 and 2. The ejector assemblies 50 are coupled to outfeed conveyors 15 that extend transversely of the conveyor system to deposit the fruit into respective bins or other collection means (not shown). The outfeed conveyors 15 are of conventional design and are not described in detail in this specification.

Each ejector assembly 50 is illustrated with particular reference to Figures 5, 6, 9 and 10. At periodic positions along the ejection section 13 of the conveyor, an ejector assembly 50 is provided to eject fruit from the roller assemblies 30. At these positions as shown in Figures 6 and 8, an ejector chain clip 60 is used to hold the ejector assemblies 50 onto the chains 20. The ejector chain clip 60 is illustrated in detail in Figure 8 and comprises a plastics moulding that has a pair of downwardly extending parallel legs 61 that terminate at their free ends in outwardly extending clip portions 62 that clip against the inside of the chain links 29 as shown in Figure 8. The legs 61 are attached at their upper ends to a pair of parallel spaced apart flanges 64 which extend transversely to one side of the legs 61. The ends 65 of the flanges opposite the legs towards its upper portion, are provided with a pair of coaxial apertures 66, which are arranged to support pins 67 that extend from either side of an ejector lever 68. The ejector clip 60 is clipped onto the chain 20 so that the parallel side flanges 64 extend inwardly of the conveyor. The ejector lever 68 is provided with an elongate ejector arm 68 that is inclined to a median portion 70 through an angle at about 135 degrees. The median portion 70 merges into a downwardly extending abutment arm 71 that extends at an angle of about 135 degrees to the median portion. As shown in Figure 5, the lever 68 is positioned so that the ejector arm 69 extends downwardly between the sides of two adjacent rollers 32 with the abutment arm 71 also extending downwardly of the pivot point defined by the pins 67. The ejector lever is arranged to pivot about the pivot point with the ejector arm 69 moving upwardly to the full line profile shown in Figure 8. This has the effect of forcing the fruit 100 on the roller 32 to roll outwardly of the conveyor, and as is shown in Figures 9 and 10 against a large rotating cylindrical brush 80 that encourages the fruit to move onto the outfeed conveyor 15 which in turn transfers the fruit to suitable collection means. As shown in Figures 9 and 10, the brushes 80 are driven through chains 151, 152 and gearing 153 to ensure that the fruit have a soft landing when they are ejected from the main conveyor onto the outfeed conveyor. The brushes 80 rotate clockwise as shown in Figure 9.

A solenoid assembly 90 that is illustrated in Figures 5 and 6 comprises a rotary solenoid 91 that is mounted on a mounting plate 92 that forms part of the aluminium section 26 that extends across the conveyor. The solenoid has a trip arm 93 of triangular cross section when viewed in plan that sits on the top of the rotary solenoid 91 and rotation of the trip arm in the horizontal plane (Figure 6) has the effect of forcing the abutment arm 71 of the ejector lever 68 to the dotted profile shown in Figures 5 and 6, this has the effect, as described above of ejecting fruit from the conveyor. The rotary solenoid assembly is activated by an electrical signal that comes from the computer.

One of the problems that can be encountered with conveyors of the kind described above is that due to the high speed of operation, i.e., fruit travelling at approximately 10 fruit per second and the comparatively light-weight componentary, there is a danger that if the ejector levers 68 do not return to the neutral position shown in dotted profile in Figure 5, the abutment arm 71 of each ejector lever 68 could, as it moves rapidly past the fixed componentary of the conveyor such as the solenoid arm 93 have a direct collison against one of the solenoid arms which could result in disintegration of the plastics componentary the effect of which would be to destroy a substantial length of the whole assembly. In appreciation of malfunctions such as a solenoid breakdown which could result in the sort of collison referred to above, a safety ramp assembly 150 is mounted on an upstanding web 151 that is secured to the aluminium extrusion 26 as shown in Figures 5 and 6. The safety ramp assembly 150 incorporates an inclined piece of tubing 153 that is inclined to the vertical as shown in Figure 5 at an angle of 45 degrees and to the horizontal through an angle of about 15 degrees as shown in Figure 6. The inclined piece of tubing 153 acts as a ramp so that in the event of the abutment arm 71 directly contacting the tube due to failing to return to the fully inoperative position shown in dotted profile in Figure 5, the ramp has the effect of forcing the arm 71 either to the left or the right of the ramp, i.e., either to the full line or dotted line positions of Figure 5 thereby avoiding the ensuing damage of a collison. Thus, the safety ramp assembly 150 might have the effect of causing one or more errors in fruit ejection but these errors are considered very slight compared with the consequences of a solenoid failure causing a direct collison with componentary of the conveyor system. Although not shown in detail in Figures 5 and 6 the actuator arm 93 of the solenoid is designed to complete its movement from the inoperative to operative positions shown in Figure 6 without impeding the safety ramp assembly 150. In the embodiment shown in Figures 1 to 10, a balanced assembly is defined whereby rollers 32 are positioned on the either end of a transversely extending drive shaft 31 that is driven via a drive wheel 98 running on a friction bed 99. The disadvantage with this assembly is that it is comparatively bulky. The embodiment shown in Figure 11 comprises a streamlined assembly whereby the rollers are axially split to define two a pair of sub-rollers 250 and 251 interconnected by a short stub axial 253 that is in turn clipped onto the chain link 29 in a manner substantially the same as in the first embodiment. An ejector assembly 50 which is also substantially similar to the first embodiment is clipped between adjacent roller sub-sets as viewed in plan and each sub-set of rollers is rotated by contact of the lowermost surface 255 of the chain clip on to a centrally positioned running surface 256 that extends along the length of the conveyor. The ejector assembly 50 operates in a slightly different manner to that described in the earlier embodiment in that the abutment arms 271 are arranged to be coupled to either side of the clip 260 and in the inoperative position assume the full line profile with the abutment arm 271 extending vertically downwardly. A solenoid mechanism 290 similar to that of the earlier embodiment is operable to cause the arm 271 to be forced inwardly to the dotted profile shown in Figure 11 whereby the other end of the ejector assembly 50 moves upwardly to eject the fruit laterally of the conveyor. The assembly also includes a safety ramp device 250 substantially similar to that of the first embodiment and the arm 293 on the solenoid 290 is arranged to act behind the abutment arm 271 to, in use, cause the inward motion of the abutment arm from the inoperative to the operative positions. This assembly has a distinct advantage that it takes up considerably less space than the assembly described in the first embodiment. The outfeed conveyors 15 and their associated ejector assemblies are arranged so that fruit of particular sizes, colours and defects can be ejected at particular points along the ejection section 13 of the conveyor. The ejection is initiated by signals that come from the computer that has interpreted the signals from the video camera, compared these signals with predetermined input data to determine the classification of sizing, colour and defect and then produces periodic electrical signals to the rotary solenoids to effect ejection at the required positions. The Classification System

The mechanical components of the conveyor system, as described above, enable fruit to be manipulated and sorted into classes of similar fruits. The following description provides a more detailed outline of how the fruit are classified into their appropriate classes.

The classification system is comprised of a computer which is connected to at least one camera. See Figure 12. The camera is positioned at a point along the conveyor system, as described above. The system described below employs only one camera, but more than one camera may be employed in order to obtain more comprehensive pictures of each piece of fruit, or to handle larger numbers of fruit at the same time.

The system is based upon the analysis of a full colour video signal from a Charged Couple Device (CCD) array camera 101. The computer includes a master processing unit which is comprised of a master processor (not shown) and a camera master processor 102. The master processor is in overall control of the classification system, whilst the camera master processor is in control of the camera(s). The camera master processor passes on the video signals to the vision processor 103 for preliminary image processing. Processed image data is then passed along the colour bus 113 to up to eight individual object processors for further processing. The object processors then pass information regarding the processed images to the master processor via the RTI-1000 bus 112. This arrangement is illustrated in Figure 12.

Figure 13 showns a two camera system. The two cameras are synchronised and produce images which provide greater coverage of the whole surface area of a piece of fruit. Alternatively, the second camera can photograph a second piece of fruit in order to increase the rate of fruit classification.

As shown in Figure 3, the camera 19 is positioned above the conveyor system 32. The fruit 100 are illuminated by the light sources 63. A number of pieces of fruit move into the field of view 71 of the camera at one time. In the present system, two rows of fruit enter the f eld of view of the camera and each row may contain up to four pieces of fruit. Consequently, up to eight pieces of fruit may be in the field of view of the camera.

The camera takes the first available photograph after the new cup sync (synchronisation) signal is received. The vision processor in the computer grabs the picture and digitises this picture. Analysis and classification of the fruit is performed using this digitised image. At this stage, up to eight pieces of fruit may be represented in the one image. Consequently, it is necessary to divide the image into eight image sections, each section containing the image of only one piece of fruit. Figure 4 illustrates the whole picture taken by the camera, and shows the eight individual image sections corresponding to each piece of fruit. Each of the image sections is sent via the colour bus to a corresponding object processor. Consequently, eight individual object processors can simultaneously process images of eight different pieces of fruit. This form of parallel processing significantly increases the rate of classification of the system. Each object processor further processes the image; this will be described further below.

Once an image has been captured by the camera, digitised, analysed and divided by the vision processor and sent to the corresponding object processors for further processing, the fruit would have moved along the conveyor system and would have rotated by a certain amount. Two pieces of fruit would have moved out of the view of the camera while two new pieces of fruit would have moved into the view of the cameras. A new picture will be taken and processed, as described above. This process continues so that up to four individual pictures of each piece of fruit are obtained. Each view should provide a different image of a different part of the surface area of the piece of fruit.

The four different views of the fruit are sent to an object processor. Similar views of different fruit are sent to other object processors, thus resulting in up to eight pieces of fruit being photographed and analysed at the same time. Once detail of the colour, grade and size of each piece of fruit is extracted by the object processor from the four views, the information is sent to the master processor by each object processor via the RTI-1000 bus.

An operator is able to specify the classification criteria to be used in sorting the fruit. For example, it may be required to sort the fruit according to colour into six different groups. Information regarding the range of colours of fruit to be sorted into each group is entered into the master processor by the operator. Upon receiving information on a piece of fruit from an object processor, the master processor compares the information on the piece of fruit with the classification criteria entered by the operator. Once the piece of fruit has been classified, it moves along the outfeed conveyor 13 and upon reaching the outlet location corresponding to its classification, it is ejected from the conveyor. The Vision Processor

The three main tasks of the vision processor are:

1. Synchronisation of the video signal with the conveyor system;

2. Digitising the video signal; and

3. Transforming the digital image into a useful form for analysis and passing onto the object processors.

Figure 13 illustrates the function of the vision processor.

The above functions will now be described in more detail. Synchronisation of the Camera and the Conveyor

The conveyor speed of the fruit lanes varies according to the required fruit flow rate. The interval between subsequent photographs, however, is constant so it is necessary to coordinate the two different rates. The rate of movement of the conveyor belt is indicated by the cup sync signal. Each time a new piece of fruit enters the view of the camera, the cup sync signal goes high. Every time the camera takes a photograph, a camera sync signal goes high. It is necessary to determine the distance travelled by the conveyor during the time between the cup sync signal and the camera sync signal in order to determine the position of the fruit within the picture.

Let the conveyor speed be in cups/second and one cup equal X metres, where for the purposes of this discussion, a cup is the length of conveyor able to handle an individual piece of fruit. Therefore, the conveyor speed is:

V = X metres/second. The elapsed time between cup sync and camera sync signals is T seconds, where the cup sync signal is a trigger signal relating to the arrival of each cup at a given location and the camera sync signal is a trigger for the next photograph. So the displacement of the image within the photograph is:

VT metres.

If the number of pixels in the images is P pixels/metre, then the displacement of the image is:

VTP pixels.

This value VTP is used to effect the image (Figure 4) to ensure that the object processors are sent data which has a fruit located centrally in the image section. Digitising the Image

The signal from the camera is an analog signal comprising the red, blue and green components of the picture. For the purposes of image processing, it is necessary to convert the analog image into a digital image. The digital image comprises a matrix of pixels, each pixel containing variable components of red, green and blue signals. The digitising of the pictures is performed by the Analog-to-Digital (A/D) convertors, shown in Figures 13 and 14. Transforming the Digital Image

At this stage the picture taken by the camera and digitised by the vision processor, still contains up to eight pieces of fruit in the one image 201. It is necessary to convert this image into a useable form. This is done by converting the digitised image into eight individual image sections 202-209, each image section containing an image of one piece of fruit. In doing this, it is necessary to use the displacement of the image information calculated in step 1, as described above. Once the images are sectioned, they may be sent to the individual object processors for further processing of the images.

The vision processor performs further transformations of the image in order to allow easy extraction of the relevant information. Before detailing the method of extracting relevant information, it is necessary to make a number of observations:

1. The fruit is being sorted on the basis of its attractiveness to humans, hence the machine should respond by sorting it in a similar way.

2. Humans have three colour senses, namely red, green and blue. People are very sensitive to green, less sensitive to red and very insensitive to blue.

3. In judging colour, the eyes use little intensity information. Therefore, intensity may be discarded during colour processing.

4. Blue component is the least significant in the colour of fruit. These characteristics are illustrated in Figure 15.

The RGB colour space may be represented in three dimensions with red, green and blue axis. Exactly where a particular coloured point belongs in the space is determined by the contributions of these three colours. An example is illustrated in Figure 16 where this chosen colour activates the red sensor, and the green sensor. The blue sensor is not activated - this colour is called yellow. This chosen point has an intensity I, which is proportional to the distance between the point and the origin in the RGB space.

This red-green-blue (RGB) interpretation, although not strictly correct in portraying the highly complex nature of colour perception, provides an extremely useful and practical model. The functions of the transformations performed by the vision processor are to manipulate the raw RGB data into a form that can be mapped onto an easier to process two-dimensional surface and to reduce the effect of light intensity on sorting results. It is important to remember that light source intensity can vary considerably while a machine is operating. Not only can the light output vary due power supply variations and light source performance can deteriorate with age but also reflected light from surfaces adjacent to the source can become dirty and less reflective.

A number of mathematical transformations on the RGB valves can largely eliminate the effect of brightness. This is done in the present embodiment with the use of a normalisation procedure, described below. It should be noted that the vision processor sets the bright and dark levels by sampling reflected light from a red, green blue and black coloured strip adjacent the fruit conveyor.

An example of one such transformer follows:

In the cartesian axis colour space the intensity I is equal to the length of the line joining the origin and the point representing the colour. Consider point A in Figure 16. Any point along the line has an equal proportion of red, green and blue so it is the same colour. Only intensity differs.

The following normalisation transformations will project all points in the colour space onto the surface of a sphere with radius K (intensity):

^R-f = R ^" where RGB are red, green, blue

I valves of given location in the

G- = G K colour space Rγ, G-γ, B-j- are

I the transformed valve for that

B ^* = B K location.

I I

K = Constant (= 1) This normalisation procedure substantially eliminates all intensity variations in the images, thus leaving images when contains substantially only colour information which can be compared with other images and classification criteria, without the interference of unwanted information, such as distortions in the images caused by light variations.

During the image grabbing procedure of the vision processor, it is necessary to capture as much of the surface area of the fruit as is possible, in order to make a correct judgement regarding classification.

As has been explained above, in the preferred embodiment, up to four images are taken of each piece of fruit. Each image is taken at a regular interval.

Between each image, the fruit move along the conveyor belt for a distance of M mm. The piece of fruit rolls along the conveyor system during this time, thus resulting in a different part of the surface of the fruit coming into the view of the camera. Between each picture the piece of fruit rolls approximately M mm along its circumference.

Since a maximum of four pictures can be taken of each piece of fruit, the maximum amount^' of surface area of the fruit that will be photographed corresponds to about 4M mm of the circumference. If the piece of fruit has a circumference less than 4M mm, it will not be necessary to use all of the image information which is photographed, thus reducing the number of images to be considered by the object processors and resulting in faster classification of the fruit.

Therefore, it is necessary to estimate the size of the fruit, in. order to estimate the number of pictures which have to be considered during the classification process.

Upon receiving the first image of a piece of fruit, the vision processor estimates the diameter, D^ , of the piece of fruit. If the diameter of the fruit is less than

4M, then it will be possible to photograph virtually all of

~~Ψ the surface of the fruit. Let T = 4M. If D, is less than T, then, the image processor takes a slice out of the image section of width M. This is shown in Figure 17A. This slice is sent to the object processor for further processing and use in the final classification procedure. When the vision processor receives the second image section, it again calculates the diameter of the object as D2« It then estimates the diameter D of the piece of fruit by averaging D-^ and D2 • It is then necessary to calculate what percentage of the second image needs to be taken into account so as to ensure that there is no overlap with the previous image if the circumference of the piece of fruit is less than 2M. The vision processor calculates the circumference of the piece of fruit as C = D1T. If C is less than 2M, then the vision processor will only take a slice of the image which is large enough to cover the whole of the circumference of the fruit, as shown in Figure 17B. This slice is aM (a<l) wide. At this stage the vision processor would have captured images covering the whole of the surface area of the fruit, and can therefore stop processing further images of that piece of fruit. Again the picture slice is sent to the object processor for further processing. If C is greater than 2M, then the vision processor will take an entire slice of width M and proceed to take and process a third image.

This process is continued until the entire surface of the piece of fruit has been photographed.

If, when receiving the first image section, the vision processor determines the diameter to be greater than T, then it will not be possible to photograph the entire surface area of the piece of fruit. Consequently, the vision processor will capture the slice of width M, as well as slice A, as shown in Figure 17C. This will give the vision processor additional coverage of the surface area. In the subsequent second and third images it only takes image slices of width M, but when it reaches the fourth image, it also takes the additional slice marked B in Figure 17D. In this way, it is possible to obtain images of virtually all of the surface area of a large piece of fruit. The Object Processors

After the vision processor has divided, normalised, and manipulated the image sections, all the required image sections for each piece of fruit are sent to the individual object processors for detailed colour, defect and size analysis. Each object processor receives the image sections of a piece of fruit, and analyses the sections according to the requirements of the classification system.

In the present embodiment, fruit are sorted according to their colour. The colour classification and selection system will now be described in more detail.

Since the majority of fruit are of a red to orange to yellow colour such as oranges, lemons, apples and peaches, the blue component in the colour of the fruit can be substantially ignored. The present algorithm maps the transformed value R._J,, G_τ, and B_p onto the two dimensional space, with G„ / IL values across the horizontal axis and B along the vertical axis. See Figure 18. The system effectively plots the transformed colour values R_™, , and B_τ for each pixel associated with the surface area of a piece of fruit onto the two-dimensional plane and totals the numbers in each of the predefined colour areas. Since the majority of fruit are in the colour range red, orange, yellow, and green, most of the plotted points would be along the horizontal axis. The more uniform the colour of the fruit, the closer the plotted points will be together. Figure 19 illustrates the fruit being divided into four classification groups, comprising red, orange, yellow and green. The master processor can be programmed to specify these different classes. As the pixels corresponding to the surface of the fruit are plotted on the plane, the master processor will select which class the fruit would fall in from the plot of its pixels.

Once this classification process by the object processor is completed and the master processor has made the decision regarding which class the piece of fruit should fall in, the pieces of fruit travel along the outfeed conveyor system and will be ejected at the corresponding ends. Size Calculations

Each of the four image sections of a piece of fruit are broken down into the long thin rectangles covering the entire surface of the fruit. By adding the volumes of cylinders of rotation of the small rectangles, it is possible to estimate the volume of the fruit using a Riemann Sum method. Using this method it is possible to then calculate the volume of the fruit as:

n = a

n = b

where: K = constant

A = first slice (first line of image) B = last slice (last line of image) D = diameter of slice (number of pixels across the slice)

These calculations are carried out about two axes at right angles to each other, as shown in Figures 20A and 20B. The lowest volume value of the two is selected and the four lowest volumes (one for each of the four views) are averaged to produce a final estimate of the volume.

The shape of the fruit can be deduced from a number of ways. Using all four image sections, diameters of the piece of fruit are measured at 45 degree angles to each other. This produces sixteen estimates of the diameter of the fruit from the four views.

In the case of substantially spherical fruit, all sixteen diameters are averaged to provide the mean diameter, see Figure 21.

For flat fruit, such as mandarines, the largest diameter for each view is selected and the average of these four values is representative of the largest circular cross section diameter of the fruit.

For long fruits, such as lemons, the smallest diameter for each view is selected and the average of these four values is representative of the smallest circular cross sections.

Additionally, the ratio of the largest to the smallest diameters from each view are averaged to produce a slenderness ratio. This ratio can be used to separate fruit of similar cross sectional diameter with largely varying lengths. For example, short almost round lemons can be packed separately from long slender lemons.

All of this colour, defect and sizing information can be compared with operator preprogrammed data to ensure that fruit of particular size and grade are consistently ejected at specific outlets.

Claims

1. Apparatus for classifying and sorting objects comprising: a singulator to form the objects into a least one row; a conveyor that is arranged to pass at least one row of the objects underneath a camera, the camera being capable of capturing at least one image of the objects; means to rotate each object whilst passing underneath the camera; an image processing means for processing of the captured image, the image processing means including means for dividing the captured image into image sections, each image section containing a representation of only one of the objects, and the image processing means in use performing further processing on each of the image sections simultaneously; and a control means for analysing output from the image processing means, classifying the objects according to predetermined criteria using the analysed output, and controlling ejection means to reject the objects at particular locations on the conveyor corresponding to the predetermined criteria.

2. Apparatus according to claim 1, wherein the singulator comprises a horizontal conveying surface bounded on either side by inclined conveying surfaces operating at a slower speed than the horizontal conveyor surface.

3. Apparatus according to either claim 1 or claim 2, wherein the conveyor comprises: a plurality of rollers spaced longitudinally in the direction of the conveyor to form at least: one row, means to drive the rollers along the conveyor, the singulator being positioned at one end of the conveyor with the inclined surfaces of the singulator extending alongside some of the rollers at the start of the conveyor.

4. Apparatus according to any one of the preceding claims, wherein the means to rotate each object comprises means to axially rotate the rollers, the object being located and supported between two adjacent rollers.

5. Apparatus according to any one of the preceding claims, wherein ejector means are provided within the gap between adjacent rollers, means being provided to periodically actuate the ejector means to cause objects positioned between adjacent rollers to be flipped off the conveyor sideways of the rollers.

6. Apparatus according to claim 5, wherein the ejector means comprises a plurality of downwardly extending abutment arms secured between adjacent rollers, means being positioned along the length of the conveyor periodically engage the arms to operate the ejector means.

7. Apparatus according to claim 6, wherein a ramp element is positioned beneath the rollers adjacent the path of the abutment arms of the ejection means whereby, in use the ejector means on contact in the ramp are displaced either side of the ramp to operative or inoperative positions respectively.

8. Apparatus according to any one of the preceding claims, wherein a pair of rollers are mounted axially spaced about a drive shaft, the drive shaft being clipped to at least one conveyor chain and the drive shaft including friction means in engagement with a fixed abutment to impart axial rotation to the shaft as the conveyor moves past the abutment.

9. Apparatus according to claim 8, wherein each shaft is clipped onto the links of spaced apart drive chains, the rollers being positioned on the outside of the drive chains.

10. Apparatus according to claim 9, wherein each ejector means is clipped onto the drive chain to extend between two adjacent rollers.

11. Apparatus according to any one of the preceding claims, wherein a plurality of outfeed conveyors are positioned along the sides of the conveyor, the outfeed conveyors being arranged to transport classified and sorted objects away from the conveyor as they are ejected from the conveyor.

12. Apparatus according to claim 11, wherein adjacent each outfeed conveyor is positioned a cylindrical brush that engages the objects as they are ejected from the conveyor and directs the objects onto the start of the outfeed conveyor.

13. Apparatus according to any one of the preceding claims, wherein the camera captures the image in colour, such that the captured image represented by red, green and blue components.

14. Apparatus according to any one of the preceding claims, wherein the image processing means includes means for digitising the captured image.

15. Apparatus according to any one of the preceding claims, wherein the image processing means includes means for normalising the captured image.

16. Apparatus according to claim 15, wherein the means for normalising the captured image includes a coloured strip for use as a reference in normalising the captured image.

17. Apparatus according to any one of the preceding claims, wherein the image processing means includes means for determining the size, shape and/or colour of each object.

18. Apparatus according to any one of the preceding claims, wherein the image processing means is comprised of a primary processing means having means for normalising and dividing the captured image, and a plurality of secondary processing means for processing the image sections simultaneously.

19. Apparatus according to claim 18, wherein the image processing means has eight secondary image processing means for processing the image sections simultaneously.

20. Apparatus substantially as hereinbefore described with reference to and as illustrated in the accompanying drawings.

21. A method of classifying and sorting objects comprising the steps of: singulating the objects into at least one row; passing at least one row of the objects on a conveyor past a camera; rotating each object whilst passing underneath the camera; capturing at least one image of each object; digitising the captured image; dividing the captured image into a plurality of image sections, each image section containing a representation of only one object; analysing the plurality of image sections simultaneously to determine information on the colour, size and/or shape of each object; classifying the objects by comparing the determined information of each object with predetermined criteria; and selectively ejecting the objects from the conveyor at stations corresponding to the predetermined criteria.

22. A method of classifying and sorting objects according to claim 21, wherein the captured image is normalised in order to substantially eliminate differences in intensity of the captured image.

23. A method of classifying and sorting objects according to claim 22, wherein the captured image further contains a representation of a coloured strip, and the normalising of the captured image is made with reference to the coloured strip.

24. A method of classifying and sorting objects according to any one of claims 21 to 23, wherein the captured image is represented in red, green and blue components.

25. A method of classifying and sorting objects according to any one of claims 21 to 24, wherein the objects are sorted according to ranges of sizes, shape, and/or colour specified in the predetermined criteria.

26. A method of classifying and sorting objects according to claim 25 when dependent on claim 24, wherein the green component is divided by the red component and plotted against the blue component, such that the ranges of colours are determined using the plotted components.

27. A method of classifying and sorting objects according to any one of claims 21 to 26, wherein more than one images are captured such that all of the surface area of each object is represented.

28. A method of classifying and sorting objects according to any one of claims 21 to 27, wherein an image is captured of eight of the objects to be sorted; the captured image is digitised; the captured image is divided into eight sections, each image section containing a representation of only one object; all eight image sections are analysed simultaneously to determine information on the colour, size and/or shape of each object; and classifying the eight objects by comparing the determined information on the colour, size and/or shape of each object with the predetermined criteria, such that the eight objects are sorted according to the predetermined criteria.

29. A method of classifying and sorting objects according to any one of claims 21 to 28, wherein first, second, third and fourth images of the eight objects are captured, and the objects are rotated in a time period between the capturing of each image, such that the images are views of different part of the surface area of each object; the first, second third and fourth captured images are successively divided into four sets of eight image sections, each image section containing one of the views of one of the objects; the eight image sections in each set are analysed simultaneously to determine information on the colour, size and shape of each object, the sets of image sections being processed successively; parts or the whole of one or more of the first, second, third and fourth views of each object are used to classify each object by comparing the determined information on the colour, size and/or shape of each object with predetermined criteria, such that the objects are sorted according to the predetermined criteria.

30. A method of classifying and sorting objects substantially as hereinbefore described with reference to and as illustrated in the accompanying drawings.