WO2018007242A1

WO2018007242A1 - Method and device for automatically placing milking cups onto the teats of a milk-producing animal

Info

Publication number: WO2018007242A1
Application number: PCT/EP2017/066191
Authority: WO
Inventors: Magnus Wiethoff; Jarek JAGODZINSKI; Clemens RENNER
Original assignee: Gea Farm Technologies Gmbh
Priority date: 2016-07-08
Filing date: 2017-06-29
Publication date: 2018-01-11
Also published as: DE102016112596A1

Abstract

The invention relates to a method for automatically placing milking cups (7) onto the teats of a milk-producing animal, more particularly a cow, comprising the following steps: - creating a two-dimensional image of the teats of the animal, wherein distance information is present for at least a plurality of image points on the two-dimensional image; - evaluating the two-dimensional image and the distance information and determining at least one position (P21) of at least one teat tip (21) of one of the teats (20) and determining an orientation of the teat (20) in a predefined coordinate system; - establishing whether the teat (20) has a skewed position; and - if no skewed position has been established, placing one of the milking cups (7) onto the position (P21) of the teat tip (21) or - if a skewed position has been established, placing the milking cup (7) at a placement position which is offset from the position (P21) by a skew value (DP) determined according to the skewed position. The invention also relates to a device for automatically placing milking cups (7) onto the teats (20) of a milk-producing animal, more particularly a cow, having a sequence control which is configured for carrying out a method of this type and which evaluates information from the sensor and controls the placement device.

Description

METHOD AND DEVICE FOR THE AUTOMATIC INSTALLATION OF MILK CUPES AT TITLES OF A MILK-LIVING ANIMAL

The invention relates to a method for automatic application of teat cups to teats of a dairy animal, in particular a cow, in which a two-dimensional (2D) image of the teats of the animal is created, wherein distance information for at least a plurality of pixels of the 2D image are determined and wherein the position of at least one teat tip of one of the teats is determined based on an evaluation of the 2D image taking into account the distance information in a predetermined coordinate system. The invention further relates to a device for automatically applying teat cups to teats of a dairy animal, in particular a cow.

In automated milking systems for dairy animals, for example for cows, the determination of the position of the teats and in particular of the teat tips of the animals is of crucial importance. Only when the position can be detected with sufficient spatial and temporal resolution, a piecing process can be performed automatically quickly and safely and thus without stress for the animal. The piecing process requires three-dimensional (3D) information about the position of the teat in space in a given coordinate system. There have been different in practice

Systems established to determine the position of the teats in the room. Mention may be made here, in particular, of imaging methods in which an initially two-dimensional image with a large number of image points (pixels) is created, which is supplemented by distance information for all or at least part of the pixels. The distance information relates, for example, to the distance between an image-capturing camera and the object reproduced by a pixel.

To obtain the distance information, for example, stereoscopic methods are commonly used, in which the distance information is derived from a distance

Comparison of two images taken from different directions can be obtained. WO 2007/104124 A1 discloses the use of an imaging time-of-flight (TOF) method as a further method, in which a two-dimensional image of the teats is carried out with the aid of only one camera, as well as distance information for each of the pixels can be obtained. Using the information about the position of the teats in the room, a device for attaching the teat cups to the teats can be controlled. In practice, robotic arms have prevailed here, which either grip individual milking cups and attach them to the teats using the position information or which move a complete milking cluster with a plurality of milking cups, wherein the milking cups can be selectively raised in order to set them one after the other. In milking parlors where automated milking is carried out, an attempt is made to bring the animal into a position that is as reproducible as possible before milking. However, one difficulty with automated milking is that the position of the teats as well as the orientation of the udder, even in one and the same animal, are very different from milking to milking. The differences have their cause eg in different posture of the animal. In addition, the size of the udder and thus the positions of the teats are also subject to daily and seasonal fluctuations.

In addition, the piecing process is made more difficult when teats project obliquely, ie not vertically downwards, from the udder. Often these teats are pushed away during an upward movement of the milking cup during the piecing process from the teat cup to the side and not detected by the teat cup. If a repetition of the piecing process fails, must be set manually, which is associated with the corresponding effort.

It is therefore an object of the present invention to provide a method for the automated application of milking cups to teats of a dairy animal, in which a piecing process can be carried out successfully even with obliquely standing teats. It is a further object to provide an apparatus for carrying out this method. This object is achieved by a method or a device with the respective features of the independent claims. Advantageous embodiments and further developments are specified in the dependent claims.

An inventive method of the type mentioned above has the following steps: In the evaluation of the two-dimensional image and the distance information, in addition to the position of at least one teat tip of one of the teats, an orientation of the teat in a predetermined coordinate system is determined. Based on the determined alignment of the teat, it is determined whether the teat has an inclination. If none Tilt is detected, one of the teat cup is attached to the position of the teat tip. If an inclination is detected, the teat cup is set at a piecing position that is offset from the position of the teat tip by an offset that is determined depending on the skew position.

By determining the orientation of the teat to be attached, the attachment position can be selectively influenced in the case of a determined inclination, in order to counteract a pushing away of the tilted teat by the teat cup. An inclination of the teat is to be understood as a deviation of the orientation of the teat from a vertical direction. It can be set a tolerance range, such that only from a predetermined deviation angle of the vertical orientation of a skew is considered to be established.

In an advantageous embodiment of the method, the alignment of the teat is described by a direction vector, which points from the teat tip in the direction of a neck of the teat on the udder. Preferably, the offset is then determined on the basis of a projection of the direction vector on a horizontal plane. For example, the teat cup can be moved from the position of the teat tip in a horizontal plane in the direction of the approach of the teat to the udder. The milking cup is then not positioned immediately under the teat tip, but rather between the neck of the teat and its tip under the teat for preparation. When lifting the milking cup at this position, the sloping teat lays over the opening of the milking cup, which is usually supplied with vacuum. The teat is then sucked into the teat cup, resulting in a successful application of the teat cups as the milking cups are raised further. In a preferred embodiment, the length of the direction vector corresponds to the length of the teat. The length of the teat can be determined in the evaluation of the 2D image taking into account the distance information. Alternatively, the length can be taken from stored animal-specific or genus-specific data. The offset in the horizontal direction can then be determined by multiplying the length of the projection of the direction vector to the horizontal plane by a predetermined factor, which is preferably greater than 0 and less than 1. The factor determines at which position the milking cup should be placed below the oblique time. With a factor of 0.5, the application is done in the middle below the teat, at a factor less than 0.5, attachment is closer to the tip of the teat, and at a factor between 0.5 and 1, attachment is closer to the attachment of the teat to the udder. Preferably, the predetermined factor is between 0.3 and 0.7, and more preferably between 0.45 and 0.55.

In a further advantageous embodiment of the method, the distance information to the plurality of pixels of a 2D image are determined substantially simultaneously. In the case of a line-by-line determination of the distance information, as occurs, for example, in (laser) scanning methods, a movement of the animal leads to an image distortion which prevents a correct determination of the skew position. The simultaneous determination of the distance information ensures that even with movement of the animal, the inclination of a teat correctly determined and the attachment position can be moved accordingly.

Preferably, the 2D image is created using a time-of-flight sensor having a plurality of pixels, wherein for each of the pixels distance information is determined in a phase detection method. This 3-dimensional mapping method meets the aforementioned criterion of simultaneously obtaining distance information and also provides it with a high refresh rate so that animal movements can be compensated during the piecing process.

In a further advantageous embodiment of the method, at least one teat cup to be attached is also detected by the 2D image of the teats of the animal, and a position of the teat cup is extracted. An application of the teat cup to one of the teats takes place taking into account a determined position of the teat in the predetermined coordinate system, as well as the determined position of the teat cup in the same coordinate system. The coordinate system can be determined, for example, by the position of the camera. Alternatively, the predetermined coordinate system can be converted based on the position of one of the teats or the milking cup so that its zero point is described by the position of one of these teats and / or the milking cup.

In this case, deviations of the determined positions of the teat cups can be compared with stored (or determined at an earlier point in time) positions. A deviation may be due to damage to the milking cluster, for example as a result of a stepping by the animal giving birth. A device according to the invention for the automatic application of milking cups to teats of a dairy animal, in particular a cow, has a piecing device and a sensor for detecting a two-dimensional image of the teats of the animal and distance information for at least a plurality of pixels of the two-dimensional image. It is distinguished by the fact that it has a sequence controller set up to carry out a method according to one of the aforementioned claims, which evaluates information from the sensor and controls the piecing device.

The invention will be explained in more detail using an exemplary embodiment with reference to figures. The figures show:

Fig. 1 is a schematic perspective view of an apparatus for automated milking;

Fig. 2 is a flow chart of one embodiment of a method for applying milking cups to teats of a dairy animal; and

3a-d a schematic representation of a piecing process of a teat cup to an obliquely standing teat in four partial images.

In Fig. 1, a part of an apparatus for automated milking is shown in a perspective view. The device comprises a robot arm 1, which can be moved in the vertical direction via an arm drive unit, not shown here, and can be pivoted about a first pivot axis 2. At a free end of the robot arm 1, a holder 3 is mounted, which can be pivoted about a second pivot axis 4 via a drive unit, also not shown here. On the holder 3, a milking cluster 5 is mounted, the milking 6 carries.

In the illustrated embodiment, the milking cluster 6 comprises four milking cups 7, which are connected via milk and vacuum tubes 8 to a milking device, also not shown here. The four milking cups 7 shown in the illustrated embodiment serve for milking cows, for example. For other animals, such as goats or sheep, a different number of, for example, two milking cups may be provided at the milking cluster. The teat cups 7 are mounted in the illustrated embodiment via a segment element 9 on the milker carrier 5. The segment element 9 comprises a plurality of annular, hollow segments, through which a traction means, for example a cable or a chain, runs in the interior of the segment element 9, which can be tensioned via an actuator. In the illustration of Figure 1, the tension elements of all segment elements 9 are stretched, whereby all four teat cups 7 are in the illustrated raised position. The traction means of the segment elements 9 can be selectively actuated via their actuators, so that each one of the teat cups 7 can be brought into a raised or lowered position.

Furthermore, an imaging sensor 10 is arranged on the cluster 5, which in the present case is designed as a TOF sensor and is correspondingly capable of producing a two-dimensional (2D) image with a predetermined number of pixels (pixels). in each case additional distance information for the distance between the image sensor 10 and the object imaged by the respective pixels is provided for the pixels. In this sense, the imaging sensor 10 can also be considered as a 3D sensor since it provides lateral and depth information. Hereinafter, the sensor 10 is for this reason also referred to as 3D sensor 10.

Around the 3D sensor 1 0 light sources 1 1 are arranged, which serve to illuminate the field of view detected by the 3D sensor 1 0. The arrangement and orientation of the 3D sensor 10 is selected such that, with a suitable orientation of the milking device carrier 5, in principle all teats of the animal to be milked can lie in the field of view of the 3D sensor 10. Apart from possible coverings of the teats, the 3D sensor 10 thus provides information from which the position and orientation of all teats of the dairy animal can be determined simultaneously. The image detail is preferably also selected such that milking cups 7 can also be detected in the raised position by the 3D sensor 10.

In the following, a method according to the invention for the automatic application of milking cups to teats of a dairy animal will be explained in connection with a flow chart shown in FIG. The method can be carried out, for example, using the apparatus shown in FIG. 1 and is explained by way of example with the reference numerals indicated in FIG. It is understood, however, that the method according to the invention can also be carried out with differently configured devices. For example, the robot arm used to move the milking cups 7 may be different from that shown in FIG. On the one hand, another movement sequence can be provided using deviating, tiltable or movable arm elements. On the other hand, for example, an embodiment of the robot arm is conceivable in which the entire milking cluster is not carried simultaneously, but in which the robotic arm grips and attaches individual milking cups. Also, the imaging sensor 10 does not necessarily have to be mounted on the robot arm or the milking cluster, but can be arranged externally and aligned with the udder area of the dairy animal. Finally, a technique other than the mentioned TOF method for acquiring the distance information may be used in addition to the 2D mapping.

For the method of attachment described below it is assumed that a dairy animal enters or has entered an automated milking parlor. Upon entering or after the animal enters the milking parlor, the animal is identified in a first step S1. For this purpose, various methods are known, for example by the animal wearing an RFI D (Radio Frequency I Dentification) mark, which is detected in the milking parlor or on entry into the milking parlor by a receiver, whereby an identification number of the animal is transmitted. In a next step S2 animal-specific data from a local or central data memory are queried based on the received identification number of the animal. The animal-specific data, also referred to below as animal data, includes information on the positioning of the animal in the milking parlor, a preferred position which the robotic arm, for example the robotic arm 1 and the milking parlor 5 assume as a standard animal-specific position and relative positional information of the animal's teats. The various information and its meaning for the present method will be explained in more detail below. In a next step S3, the animal is positioned on the basis of the received position information within the milking parlor. This can be done, for example, by placing a feed trough arranged in the milking parlor in a suitable longitudinal position. In other embodiments of the milking parlor, it may be provided that instead of the feeding trough a positive tion to bring about a lying on the rear of the animal and adjustable in position stirrup within the milking parlor in a suitable position. It should be noted that in further alternative embodiments, the step S3 can be omitted for positioning, for example, when a device for detecting the occupied animal position is present, which assumes a pre-positioning of the robot arm in subsequent process steps.

In a next step S4, the robot arm is moved to the preferred position, which was received in step S2, under the animal to be milked.

In a next step S5, a first 2D image together with distance information of the udder of the animal is recorded via the 3D sensor 10. The picture is then examined to see if the typical structures of the animal's toes are visible. For this purpose, known image analysis methods can be used, for example those for edge detection. Should it not be possible to detect structures corresponding to a teat at all, a search method can be interposed, in which the milking cluster 5 and / or the robot arm 1 are pivoted and / or tilted and / or varied in height, until the udder and the teats corresponding structure in the field of view of the imaging sensor 10 is located.

When teats can be detected in the 2D image, the number and location of a possibly detected teats are extracted from the 2D image information and the distance information provided by the 3D sensor. Should the number of structures recognized as possible teats exceed the number of teats actually present and optionally also retrieved in animal-specific manner in step S2, a corresponding number of structures are selected which correspond to the highest probability level (significance level), actually a teat , exhibit.

In a next step S6, a position of the teat tip and the orientation of the teat are determined for a first or next teat to be attached. In FIGS. 3a to 3d, an exemplary attachment process of a teat cup 7 to a teat 20 to be attached is shown in a sequence of four partial images. As shown in Fig. 3a, the teat 20 has a teat tip 21 at a position P21. The orientation of the teat 20 to be applied can be reproduced to a good approximation by a straight line and, for example, as shown in FIG. 3a, are described by a directional vector R20, which starts from the position P21 and ends at the point of attachment of the teat 20 to the udder. The length of the directional vector R20 indicates the length of the teat 20 in the present case.

It is understood that, instead of the value pair from position P21 and direction vector R20, it is also possible to determine a pair of values from the position P21 and a position of the point of attachment of the teat 20 to the udder. These different value pairs contain the same information. As will be explained below, the method according to the application can also be carried out if, in addition to the position P21, the direction of the course of the teat 20 is determined without information about the length of the teat 20 being present.

The position P21 of the teat tip 21 and the directional vector R20 describing its orientation, including the length of the teat 20 if necessary, are determined in step S6 from the 2D image information and the distance information provided by the 3D sensor.

In the case of the geometry of the robot arm 1 shown in FIG. 1 and the arrangement of the sensor 110, preferably the two rear teats are initially attached, since otherwise the milking equipment attached to the front teats conceals the rear teats. As a rear teat one of the cows usually two teats is called, which are located away from the head of the animal. With regard to their lateral position, the two posterior teats are usually located further inwards than the further spaced apart front teats.

In a subsequent step S7, it is checked, for example based on the direction vector R20, whether the teat 20 to be attached is inclined. In this case, a tolerance limit may be provided, for example an angle relative to the vertical, from which an inclination is assumed. It can e.g. be provided that only at an angle of the direction vector R20 of more than 1 0 ° relative to the vertical, the teat 20 is assumed to be inclined. If, according to these criteria, the teat 20 is not determined to be inclined, the process proceeds to a next step S8 in which the position P21 of the teat tip 21 is selected as the attachment position.

In a subsequent step S9, the teat cup 7 associated with the teat 20 found is tensioned by the segment elements 9. lifted and by means of the milking machine carrier 5 laterally and with respect to its height so moved that its opening is positioned centrally under the teat tip 21. The position of the opening of the milking cup 7 is also referred to below as the position P7 (see Fig. 3a). While vacuum is applied to the opening of the teat cup 7, the teat cup 7 is further raised, whereby the teat 20, supported by the vacuum, slides into the teat cup 7 and sucks on the teat 20 or the udder. After successful preparation, the segment elements 9 of the milking cup 7 can be relaxed, so that the milking cluster 5 can be moved to apply additional milking cups 7.

The movement of the teat cup 7 in the piecing process is preferably carried out under continuous observation by the 3D sensor 10, wherein both the position P21 of the teat tip 21 and the position P7 of the teat cup 7 are repeatedly determined from the image and the distance information.

The robot arm 1 is then driven on the basis of the relative position of teat tip 21 to teat cup 7.

In a next step S1 0 it is determined whether all milking cups 7 are attached to the number of existing teats. If so, the process ends and the milking process can begin or continue if milking of the corresponding teat has already started after successful preparation of one of the teats. If not all milking cups 7 have been set, the method branches from step S10 back to step S5, wherein again an image of the teats is picked up by the 3D sensor 10 and the position and orientation of a next teats 20 are determined in step S6. Advantageously, it is provided initially to apply the two rear teats and then only the two front teats. Whether the first or the left or the right of the front teats is first applied to each of them first depends on which of the teats is better visible.

On the other hand, if it is determined in the first or a subsequent execution in step S7 that the teat 20 to be attached is inclined, the process branches to a step S8 'instead of the step S8.

In this step S8 ', the attachment position with respect to the position P21 of the teat tip 21 is corrected by an offset Δθ determined by the orientation R20 of the teat 20. *** " Preferably, the offset ΔΡ runs in the direction the projection of the directional vector R20 on a horizontal plane. The amount of (then lateral) offset ΔΡ is preferably a predetermined portion of the length of the projection of the directional vector R20 in the horizontal plane. The predetermined proportion may preferably be 30% -70% and in particular 45% -55%. For example, if a proportion of 50% is selected, this means that the attachment position is below the inclined teat 20, in the example exactly centered between the teat tip 21 and the point of attachment of the teat 20 to the udder. The fraction can be expressed as a factor between 0 and 1, by which the length of the projected direction vector R20 is multiplied.

Thereafter, the process is continued as before with step S9, in which the teat cup 7 is moved to the predetermined attachment position and is lifted by applying vacuum to the opening.

The Ansetzposition shifted from the teat tip 21 in the direction of the starting point at the udder then leads to the situation shown in Fig. 3b, in which the inclined teat 20 is further inclined until it lies with a front portion above the opening of the teat cup 7 , At the latest in the position, it is sucked in by the vacuum at the opening of the teat cup 7 and, as shown in FIG. 3c, slides into the teat cup. A subsequent further lifting of the teat cup 7 (Fig. 3d) leads to a successful attachment. In the illustrated embodiment, the (lateral) displacement ΔΡ is dependent on the length of the projection of the directional vector R20 and thus on both the length of the teat 20 and the angle of inclination with respect to the vertical orientation. Depending on the viewing angle of the 3D sensor 10, although the inclination of the teat 20 in the region of the teat tip 21 can often be determined from the data of the 3D sensor 10, the point of attachment of the teat 20 to the udder can only be determined with great inaccuracy. It is then alternatively possible to determine the (lateral) displacement ΔΡ only on the basis of the oblique position of the teat 20, assuming a predetermined length of the teat 20. The predetermined length can be taken, for example, animal or species-specific stored data.

The correct recognition that there is any inclination at all, taking into account tolerances, if any, and also the determination of the direction of this inclination, are of importance for a successful outcome. rich piecing process. The correct recognition of the skew can be made difficult by an animal movement. Particularly in the case of 3D sensors which successively record distance data in the image area, for example in so-called SD laser scanners, movement of the animal results in image distortion which either results in an apparent skew or compensates for an actual skew. The method according to the application can therefore be advantageously used with such methods for acquiring distance data, which provide the distance data substantially simultaneously for the entire image area. Particularly advantageous in this regard is the TOF method, in which distance information for each pixel is determined at the same acquisition time and in real time with a high frame rate. Even in real time evaluated stereoscopic images can provide simultaneously detected distance information for all pixels and can therefore be used advantageously.

reference numeral

robot arm

first pivot axis

holder

second swivel axis Melkzeugträger milking

teat cups

Milk and vacuum hoses segment element

sensor

light source

teat

teat end

Position of the teat tip Directional vector Position of the milking cup Offset

S1 -S10. S8 'process steps

Claims

claims

1 . Method for automatically setting milking cups (7) on teats

(20) of a dairy animal, in particular a cow, comprising the following steps:

- creating a two-dimensional image of teats (20) of the animal, wherein there are distance information for at least a plurality of pixels of the two-dimensional image;

- evaluating the two-dimensional image and the distance information and determining at least one position (P21) of at least one teat tip (21) of one of the teats (20) and an orientation of the teat (20) in a predetermined coordinate system;

- Determining whether the teat (20) has an inclination; and

- attaching one of the teat cups (7) at the position (P21) of the teat tip (21), if an inclination is not detected, or

- Setting the teat cup (7) at a piecing position which is offset from the position (P21) of the teat tip (21) by an offset (ΔΡ), which is determined depending on the inclination, if an inclination is detected.

2. The method of claim 1, wherein the orientation of the teat (20) by a directional vector (R20) is described, which from the teat tip

(21) points in the direction of an approach of the teat (20) on the udder.

3. The method of claim 2, wherein the offset (ΔΡ) is determined based on a projection of the direction vector (R20) in a horizontal plane.

4. The method of claim 3, wherein the length of the direction vector

(R20) corresponds to the length of the teat (20) and wherein the offset (ΔΡ) is equal to the length of the projection of the directional vector (R20) in the horizontal plane multiplied by a predetermined factor.

5. The method of claim 4, wherein the predetermined factor is between 0.3 and 0.7 and preferably between 0.45 and 0.55.

6. The method according to any one of claims 3 to 5, wherein the length of the teat (20) from the two-dimensional image and the distance information is determined.

7. The method according to any one of claims 3 to 5, wherein the length of the teat (20) is predetermined animal-specific or generic individual.

8. The method according to any one of claims 1 to 7, wherein the distance information to the plurality of pixels of a 2D image are determined substantially simultaneously.

9. Method according to one of claims 1 to 9, wherein the 2D image is created using a time-of-flight sensor having a plurality of pixels, distance information being determined for each of the pixels in a phase detection method.

10. The method according to any one of claims 1 to 10, wherein in the 2D image and at least one to be applied milking cup (7) is detected and a position (P7) of the milking cup (7) is determined.

1 1. Method according to Claim 1 1, in which the teat cup (7) is set in the same coordinate system, taking into account the determined position (P21) of the teat tip (21) in the predetermined coordinate system and the determined position (P7) of the teat cup (7).

12. A device for automatic application of milking cups (7) to teats (20) of a dairy animal, in particular a cow, comprising a piecing device and a sensor (10) for detecting a two-dimensional image of the teats of the animal and distance information for at least a plurality of Pixels of the two-dimensional image, characterized in that it has a set up for carrying out a method according to one of the preceding claims sequence control that evaluates information from the sensor (10) and controls the attachment device.