WO2001076947A1

WO2001076947A1 - Method of detection of misformed package sealing clips

Info

Publication number: WO2001076947A1
Application number: PCT/AU2001/000394
Authority: WO
Inventors: Karlo Ivan Jolic
Original assignee: Chubpak Australia Pty Ltd
Priority date: 2000-04-07
Filing date: 2001-04-06
Publication date: 2001-10-18
Also published as: AUPQ677300A0

Abstract

A method of detecting misformed package sealing clips includes making a grey scale image of an affixed clip; locating contours of constant intensity to produce a plot of contour direction; determining an area of approximate clip location corresponding to a minimum standard deviation of contour direction; orienting templates of clip end sections to the average contour direction in the area of approximate clip location; determining the template locations of best fit; and comparing parameters of the template locations of best fit to those corresponding to an acceptably formed clip of the same type. The method is particularly suitable for inspecting wire twist clips applied to sausage casings.

Description

METHOD OF DETECTION OF MISFORMED PACKAGE SEALING CLIPS

FIELD OF THE INVENTION

The present invention relates to a method of detecting misformed package sealing clips and in particular, but not exclusively, to a method of detecting misformed wire-twist type clips which may for example be used to seal sausage-like packages.

BACKGROUND OF THE INVENTION

There are numerous types of product packaging available which require the use of a clip of one form or another, for sealing purposes. In particular, in the context of food packages which require hermetic sealing, the integrity of the hermetic sealing may be compromised if the clip is not formed and positioned correctly. One situation in which the method according to the present invention is of particular use is in relation to hermetically sealed "sausage-like" or roll-pack packages. Packages of this type involve the use of a flexible, usually cylindrical tubular film into which products such as minced meat, luncheon meats, sausage, salami, pet food, retort heat processed shelf stable food or the like is packed. These sausage-like packages are then sealed at their ends by the application of a wire-twist clip which takes the form of a short piece of metallic wire, usually circular in cross-section which can be bent under force to encompass the flexible film material at either end of the package. In a well formed clip of this type ends of the clip overlap and sit adjacent one another, preferably almost parallel, when formed about the flexible film material. It should be understood, however, that this type of clip is merely one example of a clip in relation to which the methods of the present invention may be adopted.

Up until the present time there has been no rapid, cost effective and qualitatively accurate means of distinguishing well formed clips from misformed clips, and thereby determining which packages are likely to experience compromised sealing. The only method of conducting this analysis previously available was by visual inspection, which is not particularly efficient or cost effective. The visual inspection task is a tiresome and monotonous one for the operator concerned, which can lead to the loss of operator concentration and compromised quality of assessment.

With these difficulties in mind, it is an object of the present invention to provide a more efficient means of detecting misformed package sealing clips. Other objects of the present invention will become apparent from the following detailed description thereof.

SUMMARY OF THE INVENTION

According to one embodiment of the present invention there is provided a method of detecting misformed package sealing clips comprising the steps of:

(i) positioning a package to which a sealing clip has been affixed so that the clip to be tested is in a standard position and orientation; (ii) illuminating the clip from a standard location with a white light source of standard intensity; (iii) taking a grey-scale image of the clip from a standard location using standard aperture and exposure time settings; (iv) analysing the grey-scale image to locate contours of substantially uniform illumination intensity and determining direction of contours to produce a plot of contour direction; (v) fitting a segment, which approximates image size of the clip, to a region of minimum standard deviation of contour direction within the plot, this region corresponding to approximate clip location; (vi) orientating templates, which approximate image size of clip end sections, to average contour direction in area of approximate clip location and fitting the templates to vicinities within the area of approximate clip location; (vii) determining template location of best fit, and comparing parameters of template location of best fit to those obtained for acceptable clip of same type, such that if location parameters are not within a predetermined threshold relative to the acceptable clip, the clip is assessed to be misformed

BRIEF DESCRIPTION OF THE FIGURES

The invention will be further descπbed with reference to the Figures wherein

Figure 1 shows a graphical representation of the means adopted for calculation of contour directions withm the grey-scale image The solid arrow withm the shaded plane represents the direction of the illumination intensity contour withm the plane,

Figure 2 shows a grey-scale image of a clip located about the flexible packaging mateπal at one end of a package,

Figure 3 shows a plot of contour direction obtained from analysis of the grey-scale image of the clip in Figure 2 The intensity at each point in the contour direction plot is directly proportional to the contour direction at the same point in the grey- scale image of the clip

Figure 4 shows a subdivision of the plot of contour direction into small box regions,

Figure 5 shows the fitting of a segment, which approximates the image size of the clip to the image shown in Figure 4,

Figure 6 shows the position of the segment located about a region of minimum standard deviation of contour direction within the plot of contour direction, which corresponds to approximate clip location,

Figure 7 shows the positioning of templates on the plot which are subdivided into smaller subsections m order to accurately locate the clip end sections, Figure 8 shows the positions of best fit for the templates, with the bar graphs in the image representing the direction counts in each template subsection, after median filtering;

Figure 9 shows a representation of the use of successively increasing image resolution and successively decreasing scan area during scanning of the template, with grid boxes representing image pixels. The circles indicate positions for which the template quality of fit is evaluated;

Figure 10 demonstrates the critical clip dimensions or parameters utilised for assessment of clip integrity, for clips of both type "A" and "B";

Figure 11 shows the appearance of type A clips which have been formed by using both excessive pressure and inadequate pressure, with the critical dimensions parameters shown;

Figure 12 shows a grey-scale image with template identifying location of clip end sections for a well formed clip (a) and a misformed clip (b).

Figure 13 the template size and orientation and the segment size and location determine the horizontal and vertical limits of the region covered by the template during accurate location of the clip.

DETAILED DESCRIPTION OF THE INVENTION

Throughout this specification and the claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" and "comprising", will be understood to imply the inclusion of a stated integer or step or group of integers or steps but not the exclusion of any other integer or step or group of integers or steps. In one embodiment the method of the invention utilises a machine vision system incorporating image processing software developed for the detection of misformed clips. The method may involve use of the following components:

• Black and white CCD camera (for example, PULNIX TM-6CN)

• Uniform white light source (for example, circular fluorescent tube, 240 V, 22 W)

• Fixture for holding the package

• Computer equipped with image-capture card • Software implementing image processing algorithm for detecting misformed clips.

The fixture for holding the package should ensure that the relative position and orientation of the clip with respect to the camera is standard or consistent between assessments conducted on different clips. The clip should appear focused in the image. The means of positioning the package may be incorporated within a mechanised package filling/sealing system or may be a separate stand into which the product is inserted. The camera aperture and shutter speed should be set to ensure the clip appears as bright as possible in the image without saturation. To ensure captured images are sufficiently bright, the clip is illuminated with a constant white light source of "standard" intensity. The position from which the image is taken, the exposure time and aperture settings should be "standard" in the sense they are consistent for assessment of all clips in order that the comparison with a well formed clip is meaningful.

The detection of misformed clips is achieved by capturing a grey-scale image of the clip and processing it with the misformed clip detection algorithm. The algorithm essentially finds the clip and calculates the critical dimensions of the clip. If the dimensions do not lie in an acceptable range, the clip is considered misformed.

From the grey-scale image it is possible to calculate and produce a contour direction plot which is an image that consists of the directions of contours of substantially uniform intensity (that is, iso-intensity lines) within the image. Assuming the end sections of the clip have a consistent profile (for example a cylindrical shape), the intensity contours of the end sections of the clip tend to have the same direction; that is, they tend to be parallel to the axis of the end sections. The end sections of the clip therefore appear as regions of approximately uniform intensity in the contour direction plot.

The contour directions may be calculated by fitting a local plane to each point in the grey- scale image and then determining the direction of the zero intensity gradient in each plane. A random real number R in the range -0.5 < R < 0.5 may be added to each intensity prior to performing the plane fit. As illustrated in Figure 1 the contour directions are calculated by fitting a local plane to at least three and preferably five near neighbour points. The direction of the intensity contour is given by the directional derivative of magnitude zero in the plane, which is represented by the solid arrow in the figure. The direction of this arrow in the X-Y plane represents the direction of the intensity contour. For planes that are parallel to the X-Y plane, the intensity contour directions are indeterminate; that is, the directional derivative of magnitude zero does not have a unique direction for such planes. Therefore, to aid the segmentation process, random contour directions are assigned to planes that are parallel to the X-Y plane.

The calculated directions (0 to 180 degrees) are represented in the contour direction plot as intensities ranging from 0 to 255. Figure 2 shows the grey-scale image of a clip, and Figure 3 shows the corresponding contour direction plot. In Figure 3 it can be observed that the cylindrical (in this case) end section of the clip appears as a region of approximately uniform intensity in the contour direction plot.

The next step is finding the approximate location of the clip. One means by which this may be achieved is by subdividing the contour direction plot into small box regions, and determining the mean and variance of the directions in each small box. In order to increase the computational speed, each pixel in the small boxes is examined only once during this calculation. Figure 4 illustrates subdivision of the contour direction plot into small box regions. A segment is then scanned across the subdivided plot such that, for every position considered, the segment encloses exactly an integer number of small boxes. That is, the size of the segment is restricted to be an integer multiple of the small box size and therefore the mean and variance of the segment can be calculated in terms of the mean and variance of the small boxes contained within it. In this way, the computational speed for determining the mean and variance of the segment is increased. Figure 5 shows how the scan positions used for the segment are defined in terms of the size and position of the small boxes. The small box size in effect determines the coarseness of the segment scan.

The position of the segment is found such that the standard deviation of the directions in the segment is a minimum which represents the approximate location of the clip. The segment size should be set approximately equal to the width of the clip, otherwise it may not find the clip if there are background image features containing a significant region of contours approximately oriented in a particular direction. Figure 6 shows the position of the segment within the plot such that the standard deviation of the directions in the segment is a minimum.

The next stage is to accurately locate the clip within the local vicinity of the approximate clip location area. This may be done by fitting templates with size roughly equivalent to the size of the end sections of the clip in the image. The templates may be subdivided into a number of smaller subsections, and scanned in the local vicinity of the segment which represents approximate clip location. The templates are oriented at the clip angle - this is obtained by calculating the average contour direction in the segment which represents approximate clip location.

The region scanned with the templates is determined by taking into account the template size and orientation, and the size and location of the segment which represents approximate clip location. Figure 7 shows a template on the contour direction plot. For each template position considered, the template quality of fit is quantified as follows:

• For each subsection, the number of directions that lie close to the calculated clip angle are counted. • The subsection direction counts are stored into an array.

• The counts in the array are then smoothed using a median filter.

• The maximum number of adjacent subsections with counts that exceed a threshold are identified. These subsections are considered to lie on the clip.

• The counts in each subsection considered to lie on the clip are then summed. This sum is used as a measure of the template quality of fit.

The template position with the highest count sum is considered to be the position of best fit for the template. At this position, the subsections considered to lie on the clip define the location and length of the clip end sections in the image. Figure 8 shows the position of best fit for the template in the grey-scale clip image. The bar graphs in Figure 8 indicate the direction count in each template subsection after median filtering. The line in each bar graph indicates the count threshold used to identify subsections that lie on the clip.

If part of the clip is occluded, this results in a lower direction count in the corresponding subsection. By setting the count threshold appropriately, these sub-rectangles can still be considered to lie on the clip. In other words, algorithm parameters can be adjusted to ensure partly occluded clips can still be accurately located.

To increase processing speed, the scanning of the template may be performed in an iterative fashion, using successively increasing image resolution and successively decreasing image area, as illustrated in Figure 9. That is, the template best fit position is obtained for the current iteration using a particular image resolution with the next iteration involving scanning the template over a smaller image area with higher image resolution. This iterative process is continued until the template is scanned pixel by pixel at full image resolution. Figure 10 illustrates the critical clip dimensions or parameters used for assessment of clip integrity. These dimensions may be calculated after the clip has been accurately located.

Figure 10 also illustrates two possible types of clip of the general wire-twist clip class, denoted by type A and type B (ie. end sections crossed over above or below relative to other end section). The definition of the critical clip dimensions is dependent on the clip type. Consideration of the clip type is necessary, otherwise it would be possible for misformed clips of type A to have the same critical dimensions as good clips of type B (and vice versa). Correct calculation of the critical clip dimensions requires prior knowledge of the clip type.

Misformed clips generally occur due to use of excessive or inadequate pressure during the clip forming process. Figure 11 illustrates the appearance of type A clips formed using excessive or inadequate pressure, and Table 1 shows the corresponding effect on the critical clip dimensions.

Table 1 - Effect of excessive and inadequate clip forming pressure on critical dimensions of type A clips

Each of the following quantities or parameters must lie within an acceptable range, as defined by predetermined upper and lower thresholds (which can be determined based upon parameters obtained for well formed clips of same type and class), in order for the clip integrity to be considered acceptable.

• a, b and c

• length of upper clip end section (a+b)

• length of lower clip end section (b+c)

• overall length of clip (a+b+c) For the method of the invention to operate optimally the axes of the end sections of the clip are required to be approximately parallel to the image plane of the camera. The diameter of the clip wire in the image, as well as the clip type and class, must be known. For best performance, the axes of the clip end sections should be oriented near the horizontal in the image. If oriented near the vertical, the clip region in the contour direction image becomes non-uniform in intensity, and accurate location of the clip is difficult.

The method is applicable to images of clips obtained using uniform white light for illumination, such that special lighting is not needed. The method does not require good image contrast between the clip and the image background so that segmentation of clips from complex backgrounds is therefore possible. The method is also insensitive to partial occlusion of the clip, and does not require the clip to have feature points, lines or special surface markings.

The present invention will now be further described with reference to the following non- limiting examples:

EXAMPLES

Example 1

Misformed Clip Detection Apparatus Used

• Black and white CCD camera PULNIX TM-6CN with Tamron F1.6 25 mm lens; distance between the camera lens and clip was 235 mm

• Circular fluorescent tube 240 V, 22 W

• Pentium III 400 MHz PC running Windows NT with 128 MB of memory • BANDIT image capture card from CORECO; captured images were of PAL format (768 by 576 pixels) with 8 bits per pixel of grey-scale resolution Algorithm Details

Misformed clip detection algorithm was written in the C programming language and implemented using MS Visual C ++ 5.0. Size of the image sub-region processed by the algorithm was 230 by 200 pixels. The total processing time was approximately 1 second per image. The images were processed using the following algorithm parameters:

• Small box size was 5 by 5 pixels

• Segment size was 50 by 50 pixels

• Template size was 105 by 52 pixels • Template was subdivided into 40 subsections

• Resolution of first template scan was 1/16^th of the image resolution

• Directions close to the clip angle satisfied the following criteria: direction (in units of intensity) > clip angle (in units of intensity) - 28 direction (in units of intensity) < clip angle (in units of intensity) + 28 • Direction count threshold was 70

• Refer to table 2 below for the predetermined upper and lower thresholds used (in units of number of template subsections). The aforementioned thresholds were obtained by calculating parameters for approximately 50 well formed clips and 50 misformed clips of the same type and class, and then using the parameters to determine upper and lower thresholds such that a maximum number of misformed clips and a minimum number of well formed clips were rejected.

Table 2 - Upper and lower thresholds

Figures 12(a) and (b) illustrate accurate identification of a good clip and a misformed clip respectively, using the misformed clip detection method. Table 3 shows the corresponding critical clip dimensions. It can be observed that each critical dimension of the good clip lies within the corresponding allowed range. For the misformed clip, however, it can be observed that critical dimensions a, b and (a + b + c) lie outside of their allowed range. Hence the misformed clip can be discerned.

Table 3 - Critical clip dimensions for clips shown in Figure 12. Dimensions are in units of number of template sub-sections

Example 2 Algorithm ' Fitting of the local plane

The local plane was defined by the following equation:

f = ax + by + c (1)

where / represents image intensity, x and y are the image coordinates, and a, b and c are coefficients that uniquely define the plane. The coefficients were calculated by minimizing the sum S defined by the following equation:

where /,^• is the intensity in the image corresponding to the image pixel with coordinates (N,-,

Yd-

The spatial distribution in the image plane of the five near-neighbour pixels used in the plane fitting satisfied the following set of equations:

X₀=x,Y₀=y

X =x + \,Y=y

X₂=x-l,Y₂=y (3)

X₃ = x, 7₃ = y + 1

X₄=x,Y₄=y-\

where (x, y) are the coordinates of the pixel of interest in the image where the plane fit is to be performed. The coefficients a, b and c were determined by partially differentiating equation (2) with respect to a, b and c, and then setting each of the derivatives to zero. The resulting set of three independent linear equations were then solved to find a, b and c.

" Determining the direction of the zero intensity gradient in each plane

The direction of the maximum gradient in each plane was obtained by applying the gradient operator to the equation of the plane:

Vf = aι +bj (4)

where i and f are unit vectors parallel to the x and y axes, respectively, of the image plane coordinate system. The direction of the zero intensity gradient in the plane is peφendicular to the direction defined by equation (4). Accordingly, the direction θ of the zero intensity gradient was found by solving the following dot product equation:

Vl.e'^θ = 0 (5)

where r = -1 . Solving equation (5) gave the following result for the direction of the zero intensity gradient in the plane (in radians):

π -< θ tan (6)

V o j

'" Calculation of the mean and variance of the segment

Let μ. and σ represent the mean and variance respectively of small box /, and let there be n small boxes of equal size that fit exactly within the segment. The mean μ and variance σ² of the segment was then calculated, in terms of μ_t and σ , by using the following equations:

^1V Determining the region scanned by the template

The region scanned by the template is defined in Figure 13. The horizontal and vertical extent of this region is dependent on the size and orientation of the template, and the size and location of the segment with minimum standard deviation of directions. Simple trigonometry was used to determine the actual limits indicated in Figure 13.

^v Directions that lie 'close' to the clip angle

Directions θ that lie close to the calculated clip angle θ_cl are defined by the following equation: θ_c,_ψ - Aθ ≤ θ ≤ θ_c!ψ + AΘ (8)

where Aθ is a constant, typically set equal to approximately 20 degrees.

^V1 Median filtering of direction counts

A median filter was applied to the subrectangle count array - that is, each element was replaced with its median value. A median filter of width three was used.

Let a[i] represents an element of the array. The following illustrates how the median of element a[i] was determined: lf(a[i - 1] >«[/]) Then

{

If( [ +1] ≤a[i\) median = a[i]

Elself(α[ + 1] >α[ι-l]) median = a[i - 1]

Else median = a[i+ 1]

}

Else

{

If( [ϊ+1] < [ι-l]) median = a[i - 1]

Elself(e[i+ 1] ≥a[i]) median = a[i]

Else median = a[i+ 1]

^vπ Template scanning using successively decreasing scan coarseness

With each successive pass of the template over the image, the region covered by the template becomes smaller while the resolution of the template scanning increases.

For example, in the first pass the template best fit position is determined using a scan coarseness of 16. This is equivalent to scanning the template across the image with a resolution equal to one sixteenth of the image resolution. The limits of this first scan are defined in Figure 13. In the second pass, a smaller image region is scanned (33 by 33 pixels in fact, centered on the best fit position of the previous pass) with a resolution equal to one eighth of the image resolution.

In the third pass, a still smaller image region is scanned (17 by 17 pixels in fact, centered on the best fit position of the previous pass) with a resolution equal to one quarter of the image resolution.

In the fourth pass, a still smaller image region is scanned (9 by 9 pixels in fact, centered on the best fit position of the previous pass) with a resolution equal to one half of the image resolution.

In the fifth and final pass, a still smaller image region is scanned (5 by 5 pixels in fact, centered on the best fit position of the previous pass) at full image resolution.

Claims

1. A method of detecting misformed package sealing clips comprising the steps of:

(i) positioning a package to which a sealing clip has been affixed so that the clip to be tested is in a standard position and orientation; (ii) illuminating the clip from a standard location with a white light source of standard intensity; (iii) taking a grey-scale image of the clip from a standard location using standard aperture and exposure time settings;

(iv) analysing the grey-scale image to locate contours of substantially uniform illumination intensity and determining direction of contours to produce a plot of contour direction; (v) fitting a segment, which approximates image size of the clip, to a region of minimum standard deviation of contour direction within the plot, this region corresponding to approximate clip location; (vi) orientating templates, which approximate image size of clip end sections, to average contour direction in area of approximate clip location and fitting the templates to vicinities within the area of approximate clip location; (vii) determining template location of best fit, and comparing parameters of template location of best fit to those obtained for acceptable clip of same type, such that if location parameters are not within a predetermined threshold relative to the acceptable clip, the clip is assessed to be misformed.

2. The method according to claim 1 wherein a stand is utilized to position package in standard position and orientation.

3. The method according to claim 1 wherein means for positioning package in standard position and orientation is incoφorated within package filling/sealing system.

4. The method according to claim 1 wherein the clip has a substantially cylindrical shape.

5. The method according to claim 1 wherein direction of uniform intensity contours are determined by fitting a local plane to at least three near neighbour points and identifying directional derivative of magnitude zone within the plane.

6. The method according to claim 5 wherein the local plane is fitted to five near neighbour points.

7. The method according to claim 1 wherein region of minimum standard deviation of contour direction within the plot is determined by subdividing the plot into box regions and determining mean and variance of direction in each box.

8. The method according to claim 7 wherein the segment encloses exactly an integer number of box regions.

9. The method according to claim 8 wherein the segment is sized to be approximately equal in width to the clip image.

10. The method according to claim 1 wherein template location of best fit is determined by:

(a) counting the number of directions that approximate angle of oriented template for each subsection of the template; (b) storing subsection direction counts into an array;

(c) smoothing counts in the array using a median filter;

(d) identifying maximum number of adjacent subsections with direction counts that exceed a threshold, which are considered to lie on the clip;

(e) summing direction counts in each subsection considered to lie on the clip, to provide a measure of template quality of fit.

11. The method according to claim 1 wherein template location of best fit determination is performed iteratively using successively increasing image resolution and decreasing image area.

12. The method according to claim 1, substantially as hereinbefore described with reference to the examples and/or figures.