WO2000011935A1

WO2000011935A1 - A method and an apparatus for locating the teats of an animal

Info

Publication number: WO2000011935A1
Application number: PCT/SE1999/001490
Authority: WO
Inventors: Mats Nilsson
Original assignee: Alfa Laval Agri Ab
Priority date: 1998-08-31
Filing date: 1999-08-31
Publication date: 2000-03-09
Also published as: SE9802923L; AU5893299A; SE9802923D0

Abstract

The present invention relates to a method and apparatus for guiding an apparatus support towards at least one teat of a milk animal comprising moving the support (32) to a position where an udder is expected to be viewed; illuminating (33) a region expected to contain at least one udder with a source of structural light from the support; and viewing captured images (34) from said support by viewing the region with an image capturing device. According to the invention, said illuminating light is alternately turned on (33) and off (35) for elimination of disturbing reflections from the captured images, said captured images (34, 36) being of two types, one type when illuminated having disturbing as well as true light reflections, and another type when light is turned off having disturbing reflections only, said captured images being analysed (37) by forming the difference between said two types of captured images resulting in a difference image, said difference image being analysed (38) to identify possible teat candidates, one of which being selected as a target teat, the position of said target teat is determined for homing (43) of said support and any supported apparatus to said target teat.

Description

A method and an apparatus for locating the teats of an animal

Technical field of the invention

The present invention relates to a method of guiding an apparatus support towards at least one teat of a milk animal, the method comprising the following steps:

- to move said support to a position where an udder is expected to be viewed;

- to iUiiminate with a source of structural light from the support a region expected to contain at least one udder; and - to capture from said support images by viewing the region with an image capturing device.

It also relates to an apparatus support guide arrangement, where a support carries a source of structured light, adapted to project at least one light line or dot on at least one teat, and an image capturing device is arranged to view said teat and to capture images thereof and to provide image signals.

Background of the invention

Such a method and such an apparatus are known from WO 97/15900.

Numerous proposals have been made for techniques by which e.g. a milking apparatus, defined as including a teat cup connected to a milling machine, a teat cleaning device and a teat examination device, can be applied to a teat of a milk animal, such as a cow, by automatic means so as to avoid the need for attendance by an operator during the procedure of milking an animal.

As automatic techniques for the rest of the milking procedure have been available for some time, automation of the teatcup application stage has become the main obstacle to development of a fully-automatic milking procedure which does not require continuous attendance and enables the so called "milking on demand" regime.

For many reasons, e.g. animal safety and comfort, milk hygiene and economic efficiency, the application stage has to be extremely reliable. That is a teat cup must be quickly and correctly applied to a teat on every occasion when an animal presents for milking. Also, the equipment to carry out the application stage has to work in difficult conditions and must be durable while not being too expensive.

A favourable solution to the problems of obtaining a fully automatic milking procedure mentioned above is described in the above-mentioned WO 97/15900. Though, obtaining effective and safe equipment for carrying out the teat cup application stage in difficult conditions, disturbing factors in some situations have had an influence on the procedure.

Thus, it has been found that when using image analyses of the captured image of the area defined by the possible teat candidates of a milking animal, reflections from other light sources, such as lamps and windows, other than the one particularly used for illuminating the said area, and from illuminated equipments in a stall, such as frames and gates, is a troublesome problem.

Object of the invention

It is an object of the invention to provide an improved teat locating technique for guiding an apparatus towards at least one teat of a milk animal.

Summary of the invention This has been achieved by a method of the initially defined kind, which is characterised in that said illuminating light is alternately turned on and off for elimination of disturbing reflections from the captured images, said captured images being of two types, one type when illuminated having disturbing as well as true light reflections, and another type when light is turned off having disturbing reflections only, said captured images being analysed by forming the difference between said two types of captured images resulting in a difference image, said difference image being analysed to identify possible teat candidates, one of which being selected as a target teat, the position of said target teat is determined for homing of said support and any supported apparatus to said target teat.

It has also been achieved by means of an apparatus of the initially defined kind, which is characterised by a control device for alternately turning on and off said light source for eliminating disturbing reflections from the captured images, said captured images being of two types, one type when illuminated having disturbing as well as true light reflections, and another type when light is turned off having disturbing reflections only, and are analysed by processing means forming the difference between said two types of captured images resulting in a difference image, said processing means further including identifying means to identify possible teat candidates, one of which being selected as a target teat, the position of said target teat is determined for homing of said support and any supported apparatus to said target teat.

By switching the illumination light on and off between different captured images, it is possible to filter out disturbing reflections and to assimilate the true reflections.

Advantageously, the forming of the difference between said two captured image types is obtained by superimposing said two captured image types in such a way that details occurring on both types are blanked out. Hereby, the method may be achieved by e.g. pure optical means or by opto-electronic means.

Preferably, the forming of the difference between said two captured image types is obtained by electronic means processing electric signals generated by said two captured images types, the signals generated by the illuminated captured image type are subtracted from the signals generated by the non-illuminated captured image type.

Advantageously, one or more teatcups carried by said support is/are attached to one or more individual target teats.

Preferably, having a fluid sprayed towards said teat candidates cleans the target teat.

Preferably, the target teat is also given a follow up treatment.

Advantageously, the illuminating light is obtained by at least one laser beam being scanned in a horizontal plane to project a light line onto said teat candidates, said light line being in turn scanned in a vertical direction, resulting in a sheet of light.

Of course, a laser beam may be scanned in a horizontal plane, resulting in a projected light line on the teat, if the scanning movement is fast enough.

Preferably, the illuminating light is obtained by two laser beams being scanned in horizontal planes, one above each other, to project two light lines onto said teat candidates, said light lines being in turn scanned in a vertical direction, resulting in two sheets of light, favourably diverging to each other. Favourably, the location technique embodying the invention uses a laser to direct at lest one generally horizontal sheet of light towards the teats of a milk animal. When viewed by the naked eye, this laser light sheet will be seen as a flickering, bright, generally horizontal line across each teat in range of the sheet and other such lines on other parts of the animal, such as the legs, if these are in range of the sheet.

An image capturing device, favourably a video camera, is directed towards the teats. The camera view axis may be in or above or below the sheet of light and is at an angle to the axis of the plane of the sheet. Conveniently, the camera looks upwardly through the sheet. The laser and the camera are fixedly related to each other and move as a whole being mounted on a milking apparatus support.

The camera receives an image generally similar to that seen with the eye. The image is then processed to recover the bright line that falls across each teat in view. (Note: the term image is used although in operation no image is displayed.) One teat from the image is chosen, conveniently the front right teat when a milking robot approaches the udder from the right front of the animal and rearwardly of the animal towards the udder, and the laser and camera moved, under control of information derived from the image processing, to approach the chosen teat by movement towards the respective bright line image on the teat.

This technique reliably distinguishes one teat from another by relative positions in the image on known positions. Similarly it can distinguish teats from background objects such as a leg or a tail or part of a stall or even debris, such as straw or other vegetation that may be in the region of the teats, by size or shape or unrealistic positions.

By making use of the alternately turning on and off of the light according to the present invention, the techniques is quite immune to stray light and different levels of ambient illumination generating disturbing reflections in the captured images, because all the disturbing reflections are filtered out and only the true reflections from the line on the teat are left and supplied to identifying means.

In one embodiment the laser and the camera are mounted on a milking apparatus support in the form of a teat-cup carrier of a milking robot, so that the robot can be controlled to approach a chosen teat closely enough for a teat-cup or other apparatus, such as a cleaning device or a following up device, on the carrier to be reliably applied to the chosen teat.

By repeated use of the technique on each remaining teat, a respective cup can be applied to each teat. Otherwise the position of the remaining teat or teats can be estimated from the established position of the chosen teat and stored information for the specific animal, including the milking history, for teat-cup application without use of the technique once the first teat-cup has been successfully applied.

The use of the term milk animal in this application is intended to be any kind of milk animals, such as cows, sheep, goats, buffaloes, and horses.

Drawing summary

Embodiments of the invention will now be described in more detail with reference to the accompanying drawings, in which:

Fig 1 illustrates a schematic diagram of an apparatus to form a teat-distinguishing image according to the invention;

Fig 2 illustrates a block diagram of the image processing hardware; Fig 3 illustrates a flow chart of the control activity;

Fig 4 and 5 illustrate representations of the images captured by the camera for respectively turned on and turned off laser device; and

Detailed description of preferred embodiments

A particular embodiment of the present invention will now be described in more detail.

Fig 1 shows a small laser 10, such as a 5-milliwatt device from Vector Technology, England, fitted with a lens 11 to provide a light sheet 12 of an included angle of some 60° to 100° with uniform illumination. Such a laser is a tube about 12 millimetres in diameter and 75 millimetres long with a fairly thin connection cable, so it is not a severe additional load for a teat-cup carrier robot arm 13, even when mounted at the outer end of the robot arm. The laser 10 is positioned to direct the sheet light 12 a small distance above the mouth 14 of a teat-cup 15 when in the carrier 16 of the robot arm 13.

A compact solid state camera 17 is also mounted on the teat-cup carrier 16. This camera is a ^lλ inch (12 mm) charge coupled device camera fitted with a lens 18 to give a 90° angle of viewing in the vertical plane. The camera is positioned at an angle to the light sheet 12, so that one extreme of the angle of view is on or below the mouth of the teat-cup 14, and therefore below the light sheet 12, while the other extreme is directed to the rear of a perpendicular to the plane of the light sheet. This positioning of the camera view angle assists in distinguishing between objects at different distances from the camera. The video performance of the camera gives a resolution of approximately 400 by 400 pixels at a 25 Hz (40 milliseconds/frame) frame rate or 400 by 200 pixels at a 50 Hz (20 milliseconds/frame) frame rate. The 90°-angle lens has a depth of shield from 300 to 50 millimetres when back focused with a small aperture (fl6). For the camera described, this resolution is about 4 pixels/degree, approximately 1 millimetre resolution at 200 millimetres with a 450-millimetre field of view.

In Fig 2 a laser stripe generator is indicated at 21 and an image capturing to view an image produced by the laser stripe is indicated at 22. A power supply unit 23 energises electronic circuit units 24 and 25 and the camera 22. The unit 25 processes the image information from the camera 22 and supplies image position information to the unit 24 which provides control information at output 27 for the robot (not shown).

The power supply unit 23 also energises the laser stripe generator 21. This energising is, however, time controlled by a timer 26 for obtaining captured images from the camera 22 when the animal (not shown) is illuminated as well as when it is not illuminated by the laser strip generator 21.

Fig 3 is a flow chart of the method of guiding a milking apparatus support towards a teat of a milk animal according to the present invention. The method begins at block 31. At block 32 the method continues with moving the support (13, Fig 1) to a fixed start position, without any reference to any animal. This fixed start position can be any suitable position, e.g. in a stable. This fixed position does not have any reference to, nor does it depend on the position of any animal. The method continues at block 33 with illuminating, by turning on a sheet of light from the laser (10, Fig 1; 21, Fig 2), a region expected to contain at least one udder. The next step, at block 34, consists of capturing from the camera (17, Fig 1; 22, Fig 2) a set of images obtained by viewing the illuminated region. Then the method continues at block 35 by having the laser turned off. By the camera another set of images is captured of the region now non-illuminated, block 36.

The captured images from the blocks 34 and 36 are compared to each other in block 37 for elimination disturbing reflections existing in both the illuminated and non- illuminated captured images as described in more detail below with reference to Fig 4 and 5. The resulting difference image obtained at block 37 is then analysed in block 38 in order to identify possible teat candidates. The next step, at block 39, consists of determining if any teat candidates have been found. If the answer is negative, i.e. no teat candidates have been found, the method continues at block 40 with moving the support the support and repeating the steps starting with block 33.

If on the other hand, the answer is affirmative the method continues at block 41. This step consists of selecting one of the teat candidates as a target teat. The method continues at block 42 with determining the position of the target teat. The next step, at block 43, consists of homing in said support and any supported apparatus to the target teat. The next step, at block 44, consists of determining if each one of the four teats of a milk animal have been selected as a target teat and the support and any supported milking apparatus have been homed in to each one of the four teats. If the answer is negative, the steps starting with block 33 are repeated. If, on the other hand, the answer is affirmative the method continues at block 45, where the method is completed.

Fig 4 and 5 show captured images as forwarded by the camera 10 (fig 1) or 22 (fig 2) when the laser 11 (fig 1) or 21 (fig 2) is turned on and turned off respectively. Referring to Figure 4, when the laser is turned on and the sheet of light is directed onto the udder of an animal, some bright image pixels 60, 61, 62, 63 and 64 are observed by the camera when the light hits different parts of the animal, such as a leg 60 and teats 61, 62, 63, 64. Also other details, such as posts of a stall (not shown) hit by the sheet of light may generate similar image pixels.

Furthermore, other light sources like windows and the general stall illumination lamps, may give some unwanted reflections in the captured images as indicated by the references 70, 71. These unwanted reflections 70, 71 are referred to as disturbing reflections when discussed below contrary to the true reflections 60, 61, 62, 63, 64 generated by the sheet of light emanating from the laser.

As is made clear by Fig 5 the disturbing reflections 70, 71 also exist when the laser is turned off.

By making use of the method according to the present invention, i.e. alternately turn on and turn off the laser, one can obtain the two different captured images as shown in Fig 4 and 5. Processing the two captured images according to the inventive method shown by the flow chart of Fig 3 and described above, the disturbing reflections 70, 71 are eliminated or cancelled as shown by Fig 6. In such a way a difference image 51 is obtained which is to be processed as described in WO 97/15900 for evaluation of the presence of a target teat, e.g. the teat 62, and to control the guidance of the robot and its carrier arm 13 (Fig 1).

Thus, the difference image 51 is examined by scanning the image pixels in columns 40 between the top and bottom (or vice versa) of the image, across the image. Moving the robot arm 13 (Fig 1) upwards and downwards performs the scanning. The laser and camera are being stationary arranged on said robot arm. The brightest pixel in each column is noted 60, 61, 62, 63, 64. When a group of such pixels side- by-side in adjacent columns and maintained in successive scans is found, this is considered as a possibility for the image of the light on the teat nearest the camera. The groups 61, 62, 63, 64 would be identified as possible teat candidates, while the group 60 would be rejected as too large for a teat, and is obviously arising from a leg. The group 62 would then be selected as a target teat, as being the nearest teat. Pixels representing reflections on hoses, parts of the stall, etc. could be rejected in a similar manner or by software techniques.

Once a suitable group 62 of bright side-by-side pixels has been identified as representing a teat of interest, a target teat, a sub-image 52 of the image portion and enclosing the identified pixel group 62 is defined, and only this sub-image is then scanned and processed to provide guidance information to the robot to move the supported teat-cup or other apparatus towards the deduced position of the pixel group.

By making use of a light source generating two sheets of light, favourably diverging to each other, it is possible to obtain a still more safe generation of captured images.

The method and apparatus according to the invention will improve the technique of teat locating for having a reliable robot guiding being safe and friendly to managing in connection with animals.

Claims

1. A method of guiding an apparatus support ( 16) towards at least one teat (62) of a milk animal, the method comprising the following steps: to move said support (16) to a position where an udder is expected to be viewed; to illuminate (10, 11; 21) with a source of structural light (12) from the support (16) a region expected to contain at least one udder; to capture from said support (16) images by viewing the region with an image capturing device (17; 22), said method being characterised in that said illuminating light (12) is alternately turned on and off for elimination of disturbing reflections (70, 71) from the captured images, said captured images being of two types, one type when illuminated having disturbing (70, 71) as well as true (60, 61, 62, 63, 64) light reflections, and another type when light is turned off having disturbing reflections (70, 71) only, said captured images being analysed by forming the difference between said two types of captured images resulting in a difference image (51), said difference image (51) being analysed to identify possible teat candidates (61, 62, 63, 64), one of which being selected as a target teat (62), the position of said target teat (62) is determined for homing of said support (16) and any supported apparatus (15) to said target teat (62).

2. A method according to claim 1, characterised in that said forming of the difference between said two captured image types is obtained by superimposing said two captured image types in such a way that details (70, 71) occurring on both types are blanked out.

3. A method according to claim 1, characterised in that said forming of the difference between said two captured image types is obtained by electronic means processing electric signals generated by said two captured images types, the signals generated by the illuminated captured image type are subtracted from the signals generated by the non-illuminated captured image type.

4. A method according to anyone of preceding claims, characterised in that one or more teatcups (15) carried by said support (16) is/are attached to one or more individual target teats (61, 62, 63, 64).

5. A method according to anyone of preceding claims, characterised in that said target teat (61, 62, 63, 64) is cleaned by having a fluid sprayed towards said teat candidates.

6. A method according to anyone of preceding claims, characterised in that said target teat (61, 62, 63, 64) is given a follow up treatment.

7. A method according to anyone of preceding claims, characterised in that said illuminating light (12) is obtained by at least one laser beam being scanned in a horizontal plane to project a light line onto said teat candidates (61, 62, 63, 64); and said light line being in turn scanned in a vertical direction (80), resulting in a sheet of light.

8. A method according to anyone of the claims 1 to 6, characterised in that said illuminating light (12) is obtained by two laser beams being scanned in horizontal planes, one above each other, to project two light lines onto said teat candidates (61, 62, 63, 64); and said light lines being in turn scanned in a vertical direction (80), resulting in two sheets of light, eventually diverging to each other.

9. An apparatus support guide arrangement, where a support (16) carries a source (10, 11 ; 21) of structured light (12), adapted to project at least one light line or dot on at least one teat (61, 62, 63, 64), and an image capturing device (17; 22) is arranged to view said teat (61, 62, 63, 64) and to capture images thereof and to provide image signals, said apparatus being characterised by a control device (26) for alternately turning on and off said light source

(10, 11 ; 21) for eliminating disturbing reflections (70, 71) from the captured images; said captured images being of two types, one type when illuminated having disturbing (70, 71) as well as true (60, 61, 62, 63, 64) light reflections, and another type when light is turned off having disturbing reflections (70, 71) only, and are analysed by processing means (25) forming the difference between said two types of captured images resulting in a difference image (51); said processing means (25) further including identifying means to identify possible teat candidates (61, 62, 63, 64), one of which being selected as a target teat (62), and the position of said target teat (62) is determined for homing of said support (16) and any supported apparatus (15) to said target teat (62).

10. An apparatus according to claim 9, characterised in that said processing means (25) is arranged to superimpose said two types of captured images for blanking out details occurring on both said image types.

11. An apparatus according to claim 9, characterised in that said processing means (25) comprises electronic means processing electric signals generated by said two captured images types; and the signals generated by the illuminated captured image type are subtracted from the signals generated by the non-illuminated captured image type.

12. An apparatus according to anyone of the claims 9 to 11, characterised in that one or more teatcups (15) is/are carried by said support (16) to be brought in contact with respective one of said teat candidates (61, 62, 63, 64).

13. An apparatus according to anyone of the claim 9 to 12, characterised in that a cleaning device is attached to said support (16), said cleaning device being arranged to spray said teat candidates or target candidate (61, 62, 63, 64) with a cleaning fluid.

14. An apparatus according to anyone of the claims 9 to 13, characterised in that support (16) comprises an apparatus for following up treatment of the teat candidates (61, 62, 63, 64).

15. An apparatus according to anyone of the claims 9 to 14, characterised in that said light source (10, 11 ; 21 ) is of laser type generating at least one beam scanned by a scanning device in a horizontal plane to project a light line onto said teat candidates (61, 62, 63, 64); and said scanning device also scanning said light line in a vertical direction (80), resulting in a sheet of light.

16. An apparatus according to anyone of the claims 9 to 14, characterised in that said light source (10, 11; 21) is of laser type generating two beams scanned by a scanning device in horizontal planes, one above each other, to project two light lines onto said teat candidates (61, 62, 63, 64); and said scanning device also scanning said light lines in a vertical direction, resulting in two sheets of light, favourably diverging to each other.