NL2009923C2

NL2009923C2 - System for detecting and determining positions of animal parts.

Info

Publication number: NL2009923C2
Application number: NL2009923A
Authority: NL
Inventors: Assaf Wise; Gideon Yadin
Original assignee: Lely Patent Nv
Priority date: 2012-07-20
Filing date: 2012-12-03
Publication date: 2014-01-23

Description

System for detecting and determining positions of animal parts

The present invention relates to an animal part detection system for detecting a part of an animal in a part of space in which said animal part is expected, and determining a position of said animal part, and comprising an optical camera for 5 collecting optical images of said part of space, a processing unit connected to the camera arranged to process said optical images, wherein the processing unit is further provided with at least one criterion for detecting said part of an animal in said part of space by means of said shape information and/or distance information.

10 Such systems are known in the art, for example to couple extraction devices to lactiferous organs of lactating mammals, in the form of an ultrasound camera or the like.

In practice, such systems not always function satisfactorily, either because they 15 are to slow or too unreliable. This is undesirable in that it reduces the capacity of any machine that uses such a system for operation. For example, if the extraction devices cannot be coupled to the lactiferous organs, effective production time is reduced, and furthermore discomfort for the animal arises. Even if the operator would be warned and connect manually, the waiting time and the manual labour 20 are undesirable.

It is therefore an object of the present invention to provide a system of the kind mentioned above, that is more reliable and/or faster, or at least provides the public with a reasonable alternative.

25

The invention achieves this object with a system according to claim 1, in particular an animal part detection system for detecting a part of an animal in a part of space in which said animal part is expected, and determining a position of said animal part, and comprising an optical emitter for emitting a bundle of optical radiation into 30 said part of space, a first device arranged to cause the bundle to have an arrangement of spatially regularly or irregularly alternating regions of higher and lower radiation intensity with respect to one another, an optical camera for collecting optical images of said part of space, a processing unit connected to the 2 optical camera arranged to process said optical images by comparing one of said optical images with at least one reference image of said arrangement and collected by said optical camera, and cross-correlating said arrangement as collected in said optical image with said arrangement as collected in said 5 reference image, thereby providing shape information and/or distance information, wherein the processing unit is further provided with at least one criterion for detecting said part of an animal in said part of space by means of said shape information and/or distance information.

10 With such a system, use is made of the way in which objects distort a known arrangement. By evaluating the local distortion, e.g. the mutual distance of neighbouring parts in the arrangement or their size, one can extract information about the distance and orientation of surfaces and structures in the collected image. For example, in the simplest case of a completely regular square 15 arrangement emitted in substantially parallel bundles, portions of the image in which the inter-neighbour distance is relatively larger but still regular are closer to the camera than parts with a regular but relatively smaller. In a similar fashion, parts in which the individual features are "compressed" in one direction are determined to be tilted in that direction.

20

Such systems are found to be very precise and fast, due in part to the high resolution available for optical camera and the availability of fast and precise algorithm to process optical images. Herein, "optical" is meant to comprise "visual" as well as NIR (rounded off 1 pm) and ultraviolet. Also, use is made of a single 25 camera and a single image, though in processing it is compared to one or more previously taken and stored reference images. But since the processing can be done by an external and high power computer, no time is lost in taking two or more images before processing can begin, as is the case for e.g. triangulation based systems. Contrarily to the latter, the present invention allows a reliable and fast 30 processing, which is useful to deal with unpredictably moving animals, which is a big difference from non-moving inanimate objects, or inanimate objects that are moved by e.g. an endless belt, i.e. in a completely predictable fashion.

3

Depending on the objects to be detected, a number of criteria may be applied to the image. For example, if a lactiferous organ has to be detected with a bovine, one can look for a more or less cylindrical object with a diameter of about 2-3 cm and a length roughly between 2 and 8 cm, with a rounded tip to the lower side and 5 connected at the upper side to a much bigger spherical structure, and moreover being provided in fourfold in a trapezoidal symmetry. Of course, if other objects need to be detected, suitable criteria can be provided, based on knowledge of the geometry of those objects.

10 Particular embodiments and/or advantages are described in the dependent claims and in what follows.

The optical emitter is advantageously a source of coherent radiation, such as a laser or laser diode. Due to the coherence, an arrangement induced in the bundle 15 will be present over an appreciable distance, such that use can be made thereof in detecting structures in a part of space.

Also, the optical emitter can comprise for example an LED, combined with suitable optics to create a substantially parallel bundle of radiation. Such an LED could be 20 a colour LED or even a white or wide band LED, in some cases provided with a filter for wavelength(s) selection.

Preferably, the optical emitter is tunable having a selectable wavelength for the emitted radiation. Herein, "selectable wavelength" relates to either the weighted 25 average value FWHM, or the central wavelength value of the emitted radiation. Thus tunability can be achieved by tuning the very source, such as a tunable laser, and or by inserting an optically active element, such as a variable filter, or a collection of filters with different pass bands, that can be selected to be positioned in the bundle. A great advantage of such tunability is that it is possible to better 30 separate the operation of a system according to the present invention from a similar system when present in the same room. For example, if a machine with extraction devices and a system according to the present invention is present in an animal housing, as well as an automatic device for moving excrements with such a system, their respective cameras might collect images originating from the other's 4 emitter. Although it is possible to use synching, or mutually attuned time frames for taking images, this requires some form of communication, which is cumbersome. In the case of a tunable emitter, a simple selection of the wavelength (or filter or the like) upon introduction of an additional system will suffice, which is much 5 easier.

According to the invention, the first device is arranged to impart a regular or irregular arrangement of spatially regularly or irregularly alternating regions of higher and lower radiation intensity with respect to one another. This means that 10 the bundle of radiation shows a varying intensity in a plane transverse with respect to the bundle, such as a patched, chequered, spotted or tessellated bundle. Thereto, the first device could be a piece of transparent material, provided with a suitable arrangement, such as ground glass. Here, the differences in bundle intensity arise due to interference effects for the various bundle parts after the 15 glass. The first device could also be or comprise a thin layer of transparent material that has been provided with a collection of opaque and less opaque areas, and in a regular or irregular order. Although a regular order, such as a square order, would greatly simplify calculations, it would be more difficult, or rather impossible, to discern different regions from one another. Therefore, 20 advantageously, the arrangement is a non-repetitive arrangement of irregularly alternating regions of higher and lower intensity with respect to one another. For more details regarding the first device, reference is made to US Appl. No. 11/899,542 to Freedman et al., and to US Appl. No. 60/724,903 to Zalevsky et al., in particular 4-5, each of which documents are hereby incorporated by reference.

25

In particular embodiments, the part of the animal is a lactiferous organ of the animal, such as the four lactiferous organs of a bovine. Such animal parts have a high importance in the agricultural industry, while they move in highly unpredictable ways, not only because of the general movements by the animal, 30 but also because the physical connection between the lactiferous organs and the bovine as a whole is rather loose and allows relatively large swings.

As mentioned, the system works well in respect of unpredictably moving objects, such as animals and more in particular lactiferous organs. Preferably, the system 5 further comprises an animal operating device controlled by the processing unit on the basis of the provided shape and/or distance information, in particular on the detected animal part, more in particular on the detected lactiferous organ. Especially in the case of repeatedly detecting the lactiferous organ(s) and its 5 (their) position(s), the animal operating device can be guided towards these quickly and reliably, thereby improving the capacity of the system as a whole. Examples of the animal operating device comprise extraction devices for connection to the lactiferous organs, as mentioned before, but also cleaning tools such as a brush or a spray nozzle. Nevertheless, it will be clear that the present 10 system is also applicable for other animal parts, such as snouts, ears or legs.

The invention will now be explained in more detail by means of exemplary and non-limiting embodiments in the drawings, in which: 15 - Figure 1 diagrammatically shows a system according to the invention; - Figure 2 diagrammatically shows another system according to the invention; and - Figures 3A and 3B diagrammatically show a part of a reference image, and of an actual image, respectively.

20 Figure 1 diagrammatically shows a system 1 according to the invention, having an optical emitter 2, a camera 10 and a processing unit 20.

The optical emitter 2 comprises a laser 3, emitting a laser bundle 4. Reference numeral 5 denotes a piece of ground glass. Reference numeral 6 denotes a 25 diffractive element. The bundle emitted is denoted 7, shining on an object 8. Item 9 denotes a variable filter.

The camera 10 has a camera lens 11, a ccd 12, and an effective field of view 13.

30 The processing unit 20 has a processor 21 and a memory 22.

The laser 3, that can be any known laser, such as a solid-state laser or gas laser, emits a laser bundle 4 of a suitable colour or wavelength. These could be for example visual wavelengths, although near infrared wavelengths, such as around 6 800-1000 nm are sometimes advantageous because they do not hinder most animals. The piece of ground glass 5 acts as the first device, for providing the arrangement of higher and lower intensity regions in the bundle to be emitted. The bundle of light is now sent through a diffractive element 6, that helps to control the 5 brightness level in a direction transverse to the bundle propagation direction. Subsequently, the bundle is arranged as bundle 7, having the arrangement of spatially regularly or irregularly alternating regions of higher and lower radiation intensity with respect to one another. The piece of ground glass may cause the bundle to have an irregular arrangement of such regions. The piece of ground 10 glass 5 could also be replaced by a transparency with a suitable arrangement imprinted thereon, while the laser 3 could then be provided with a bundle expander, or be replaced by e.g. an LED with a lens. For more information, reference is made to US Appl. No. 11/899,542 to Freedman et al. Item 9 denotes an (optional) variable filter, that can be used for tuning or selecting a wavelength 15 from the bundle emitted by the source of radiation, if its bandwidth is sufficient. In case it is a tunable laser, the laser 3 itself will be (slightly) variable as to wavelength. In other cases, laser 3 will be replaced by a LED, such as a white LED, either a multiband LED or a phosphor coated blue light LED, the latter allowing various broader pass bands, thus allowing a better signal separation and 20 signal-to-noise ratio. If two or more systems according to the invention are working close together, the images of the systems might overlap, causing disturbances or even malfunctions. To separate the operation of the system, each may be selected to have its own wavelength.

25 Then, when the bundle with the arrangement hits an object 8, a part of the bundle will be reflected and be recorded by the camera 10, in which the camera lens 11 forms an image on the ccd 12. The camera 10 could contain a filter, for filtering out the emitted wavelength, in order to minimise the effect of ambient light. The image on the ccd is then sent to and processed by the processing unit 20, in which it is 30 compared to the processor 21 to a reference image stored in memory 22. This will be elucidated in more detail in connection with Figures 3A and B. In practice, it will be useful to fixedly connect the optical emitter 2 and the camera 10 to one another, so that reference images need only be taken once. Contrarily, the 7 processing unit 20 may be positioned anywhere, as long as it is operatively connect to the camera 10.

Figure 2 diagrammatically shows another system 30 according to the invention. 5 The system shows a space 40 for treating a bovine 41, in this case by connecting extraction devices 32 to lactiferous organs 43 of the uber 42, by means of an articulated arm 31 controlled by processing unit 20.

In this example, the processing unit 20 will image and detect in particular the 10 lactiferous organs 43 in the image, by means of cross-correlating a distorted arrangement of lower and higher intensity parts thereof with a similar arrangement in a reference image, as has been crudely outlined above, and will be explained more clearly further below. The processing unit can further be arranged to detect the one or more extraction devices 32 in the image, so that e.g. it is able to guide 15 the latter to the lactiferous organs for connection, by minimising the distance there between. As the bovine 41 may move unexpectedly, the position of the organs may change similarly unpredictably and rather abruptly. With the faster imaging and detection system according to the invention, one is better able to follow the position of the organs and connect the extraction devices faster and more reliably. 20 Of course, other devices than the extraction devices may be used, such as brushes, just as well as other animals than bovines may be treated, however mostly (female) ungulates.

Figure 3A diagrammatically shows a part of a reference image, and Figure 3B 25 diagrammatically shows a part of an actual image taken by the camera 10.

The reference image of Figure 3A is an image of the bundle 7 taken at a known distance. The image shows the arrangement of the alternating regions of higher and lower radiation intensity as present in space at said distance. Just for 30 convenience, the arrangement is shown as completely regular. This greatly simplifies the following discussion. However, it is to be noticed that a random, non-repetitive arrangement is much more convenient in practice, as this allows to identify a part of the actual image with much easier and with more certainty. Furthermore, although circles have been indicated in Figure 3, this does not mean 8 that there are only bright spots while all the rest is dark. Rather, the circles indicate brighter parts in the image, while the parts around and in between the circles is darker, but not necessarily completely dark, even without a view to ambient light.

5 The actual image 3B, highly idealised in this case, shows how the emitted bundle 7 would be imaged when illuminating a part of a bovine. One can see an arrangement of circles and more or less circles-like spots in the image. Some parts are lacking circles and spots, obviously the parts where no reflection takes place. Furthermore, some parts do show an arrangement, that has, however, been 10 distorted with respect to the original. The distortion of the arrangement, and in particular the distance between neighbouring circles or spots, and also the (average) size of the circles or spots is an indication of the orientation and the distance with respect to the camera of the surface reflecting the arrangement, but can also be also compared with the distance at which the reference image 3A was 15 taken. For example, a part of the arrangement slightly above the centre of Figure 3B shows circles at about the same distance as in Figure 3A, and also in about a square arrangement. This indicates that the reflecting surface is oriented substantially transversely with respect to the camera and at about the same distance as for image 3A. To the left and right thereof, the circles are more and 20 more closer together, and run off to the top of the page. This indicates that the surface bends further away, i.e. bends to the back, and furthermore is slightly inclined such as to face the ground. In all, the central part of the image seems to resemble roughly a semi-circle, better: a half-sphere. Looking more closely, four structures can be found having a more or less cylindrical shape with a rounded tip. 25 These are obviously the bovine lactiferous organs. To the extreme left and right edges of the Figure 3B, similar cylindrical structures can be seen, which can be recognised as legs, while the large structure at the top of the Figure will be the abdomen. Note that in this case the image analysis is a kind of two-step analysis. First, a three-dimensional image is created by determining, for as many 30 circles/spots as possible, the spatial coordinates thereof. Then, the three-dimensional image is further analysed in order to extract surfaces and shapes therefrom, by means of image and shape recognition techniques. These are deemed known per se to the skilled person.

9

It will be clear that in the above Figure 3B, the picture is much clearer than will be the case in practice. For example, there is no dirt, noise or background signal present, the structures are all physically separated and easily recognisable. On the other hand, it is more difficult to actually position the structures in space (i.e. 5 determine the right orientation and distance) with a completely regular arrangement, as it is not possible to distinguish parts in a repetitive arrangement. Thereto, an irregular arrangement is used, for example random or regular though non-repetitive arrangements. With such arrangements, it is easier to crosscorrelate parts of the collected image and similar parts of the reference image, as 10 no so-called wrapping problem occurs. Reference is again made to W02007/043036.

The above embodiments and drawings are not intended to limit the invention, the scope of which is determined by the appended claims.

15

Claims

An animal part detection assembly (1) for detecting a part (8) of an animal in a space part in which said animal part is expected, and for determining a position of the animal part, and comprising 5. an optical emitter (2) for emitting a bundle (7) optical radiation to the space portion - a first device (5) arranged to make the bundle have an arrangement of spatially regular or irregularly alternating regions of relative higher and lower radiation intensity 10. a camera (10) for collecting of images of the space portion - a processing unit (20) connected to the camera and arranged to process the images by: - comparing one of the images with at least one reference image of the arrangement and collected by the camera, and - cross-correlating the arrangement as collected in the image with the arrangement as collected in the reference image, thereby providing shape and / or distance information, wherein the processing unit is further provided with at least one criterion for detecting the animal part in the space part by means of the shape and / or distance information.

An assembly according to claim 1, wherein the optical emitter is a source of coherent radiation.

3. Assembly as claimed in claim 1 or 2, wherein the optical emitter comprises, for example, an LED, combined with optics suitable for providing a substantially parallel beam of radiation.

An assembly according to any preceding claim, wherein the optical emitter is adjustable with an adjustable wavelength of the emitted radiation.

5. Assembly according to any preceding claim, wherein the first device is a piece of transparent material, provided with a suitable arrangement, such as ground glass, or wherein the first device is or comprises a thin layer of transparent material, which is in a regular or irregular pattern. is provided with a collection of opaque or less transparent areas.

An assembly according to any preceding claim, wherein the arrangement is a non-repetitive arrangement of irregularly alternating regions of higher and lower intensity relative to each other.

An assembly according to any preceding claim, wherein the animal part 5 is a milk-carrying member of the animal.

Assembly as claimed in claim 7, further comprising an animal processing device controlled by the processing unit on the basis of the provided shape and / or distance information, in particular on the detected animal part, more in particular on the detected milk-carrying member.