GB2310100A - Labelling objects in image processing sysstems - Google Patents

Abstract

An image labelling system (10) processes an acquired image and processes the image to provide 1 bit pixels. The connectivity of each pixel is examined by decision-making apparatus (18) and labels associated with each pixel supplied by a labeller (20). The labels are determined by the labels previously applied to the pixels connected to that being labelled. The labeller (20) sends equivalence messages to processor (17) when a pixel has connecting pixels with different labels and assigns to the pixel the smallest label of that of its neighbours. A global labeller (38) is arranged to re-label the image so that each connecting pixel in one object in the image has the same label.

Description

IMAGE LABELLING SYSTEM This invention relates to an image labelling system. It particularly concerns a system used for labelling objects in image processing systems.

Connected component labelling (CCL) is an operation used in image processing systems for segmenting binary images into regions and assigning unique labels to each unconnected region. A picture of an object can be digitised and thresholded and labelled by CCL to distinguish that object in the picture from any other objects Typically each pixel within the object is assigned the label for that object. Any other objects detected are then assigned separate and unique labels. Each region or object may then be characterised by measuring parameters such as size. position of centroid or co-ordinates of bounding rectangle. Typically, these characteristics are passed to processing functions for making high level decisions about objects within the image, such as recognition and classification.

For applications such as tracking or surveillance of objects the response time to detection of an object or changes to an object are critical. This is particularly true where output from the processing of an acquired image is used in a closed loop to control a system. Response times must therefore be as short as possible and labelling and processing must be carried out with minimum delay. As a consequence of this there has been a significant amount of research directed towards the implementation of CCL in real time. Much of this research has been concerned with devising fast CCL algorithms.

The high processing speeds required suggest that parallel processing arrays should be used for CCL. However, CCL has properties which make parallel processing implementation difficult. Two pixels may form part of the same object, and hence require the same label, but they may be spatially separated by a large distance.

Additionally, the shape made by the interconnecting pixels between them may be highly convoluted. This means that there must be some form of global communication between arbitrary pixels in an image to enable them to be allocated the same label if necessary. However, if CCL is performed on parallel processing systems where there is limited connectivity between processing nodes, then there is likely to be communication problems arising which are non-deterministic, i.e. data dependent in nature.

Investigations of algorithms for CCL have investigated techniques for partitioning images geometrically and processing each partitioned image on a separate processor. However, this leads to difficulties where an object extends across boundaries of partitions, because labels assigned to each individual image need to be unified across the image as a whole. Such operations can only be performed globally across images as a whole, and consequently represent a bottleneck in the parallel processing. Typically, for an image frame of 256 x 256 pixels labelling takes 400 ms.

In real time systems operating at 25 frames per second this equates to a delay of 10 frames, which is unacceptable for typical surveillance and tracking applications. For larger frames of 512 x 512 the labelling period is at least four times as long, i.e. 40 frames.

Two real time CCL algorithms were described by A. F. Lochovsky in Algorithms for Real Time Component Labelling of Images" (Image Vision and Computing, Volume 6, No. 1, pages 21 to 27, February, 1988). However, these algorithms are designed to be implemented in custom hardware and are inappropriate for use in currently available commercial hardware. They involve the use of a purging" operation which is required to re-set and re-use internal data structures within the algorithms. This has the disadvantage that the labelling process is interrupted by the purging operations. The purging operations are intended to be carried out in the gap between lines in the image, but in typical commercial systems the gap is only of the order of microseconds. This is too short to carry out these purging operations. In systems where the line gap is sufficient for purging operations the resultant time lag causes further processing delay.

Hambrusch and TeWinkel ("A Study of Connected Component Labelling Algorithms on the MPP", Proceedings of the 3rd International Conference on Supercomputers, Boston, 15-22 May, 1988, Volume 1, pages 477 to 483) described a number of CCL algorithms for use on the Massively Parallel Processor (MPP) operated by National Aeronautics and Space Administration (NASA). The MPP is a single instruction multiple data (SIMD) machine, with a 128 x 128 mesh of processing elements, each of which is a bit serial processor with 1024 bits of storage. The algorithm's optimum operation is when the image size corresponds to the size of the mesh. The fastest algorithm has processing times of between 26 ms and 72 ms, depending on the test image used. Images larger than the mesh must be partitioned geometrically, with each patch within a partition being individually labelled. The labels then have to be reallocated on a global basis for the image as a whole. Consequently for a 512 x 512 pixel image the processing time is at least sixteen times as long as for the smaller mesh-sized test images, making the technique inappropriate for use where real time processing is required.

The firm Sumitomo Metals manufactures two devices identified as lP90C1 0 and IP9OC11 Labelling Accelerators. The latter is described in the Sumitomo Metals sales literature as being a lower cost version of the former, the lower cost being achieved by reducing the number of labels and reducing the operating frequency.

The sales literature states that real time labelling can be achieved using three of either of the devices. For each of the three devices labelling requires three frames: one frame for each of the functions of primary labelling1 label unification and final labelling. With three devices operating in parallel each of the functions can be carried out in one frame. The disadvantage is that hardware costs are increased because of the extra devices required for real time implementation.

It is an object of the invention to provide an alternative image labelling system.

The present invention provides an image labelling system for labelling objects in an image comprising a plurality of pixels, the objects comprising at least one foreground pixel, wherein the system comprises: (i) means for identifying each pixel of an image as either a foreground pixel or a background pixel; (ii) local labelling means for assigning a local label to foreground pixels such that connected foreground pixels may receive the same local label and for generating equivalence messages when a pixel within an object has neighbouring pixels with differing local labels; (iii) global labelling means for assigning global labels to objects in the image such that objects comprising a plurality of pixels with differing local labels are assigned a single global label; and (iv) equivalence processing means for receiving the equivalence messages from the local labelling means and for determining which locally labelled pixels correspond to the same object; and wherein the local labelling means and the equivalence processing means are arranged such that the operation of the local labelling means is not delayed by the operation of the equivalence processing means.

In general, the labelled image may be generated immediately after the input image has been scanned, with no pause or latency. For some highly complex images, this will not be the case, and there will be a short, data dependent latency between the end of the image input and the start of the result output. However, this latency will still, in general, be shorter than with alternative schemes.

Whilst the image is being labelled, certain parameters may be calculated for each object, for example area and centroid co-ordinates. The component labeller of the invention has the capability to exclude from the output image objects whose parameters exceed, or fail to exceed, certain threshold criteria. For example, small objects below a threshold area value may be excluded. This has advantages over alternative techniques which first label all objects then remove certain objects from the image because it economises on the use of label values and typically it will be faster. This operation is carried out at the expense of some latency. Although the parameters for each object are computed concurrently with the labelling operation, the comparison against thresholds and the decision as to which objects to retain requires some latency between input and output. This latency is data dependent.

In order that the invention might be more fully understood, embodiments thereof will now be described, by way of example only, with reference to the accompanying drawings in which: Figure 1 is a schematic diagram of an image labelling system of the invention; Figure 2 is a schematic diagram of a window for establishing connections between pixels; Figure 3 is a schematic of an object in an image which has been assigned two different local labels; Figure 4 is two diagrams of (a) an input image containing a number of objects and (b) the object with local labels assigned to their constituent pixels; Figure 5 is a schematic of the tree structures used in the assignment of local labels to the pixels of Figure 4.

Referring to Figure 1, there is shown a schematic diagram of an image labelling system of the invention indicated generally by 10. The system 10 has a video camera 12 which is a Pulnix TM6CN monochrome CCIR device operating at 25 frames per second. Output from the video is input to an analogue/digital converter 14 which is a KSV3110 analogue to digital converter with a SAA1034 video sync.

generator to extract video timing from the video signal. The analogue/digital converter 14 resolves each input frame into a 512 x 512 pixel frame, with the intensity of each pixel being represented by an 8 bit code.

The 8 bits from each pixel are passed to a pre-processor 16 which is a preprogrammed Altera programmable logic device. The pre-processor 16 converts the 8 bits of each pixel to a 1 bit value. The bit value assigned to a pixel is 1 if it is a foreground, i.e. object, pixel and 0 if it is a background pixel. The pre-processor 16 operates in one of two modes. In arithmetic mode, it compares the grey level value of each pixel against a lower and upper threshold, programmable by a transputer 17, using either signed or unsigned arithmetic. If the pixel grey level value is greater than or equal to the lower threshold and less than or equal to the upper threshold then the output is 1, otherwise it is 0.

In logical mode, for each bit of the pixel grey value, an ExclusiveOR (XOR) operation is carried out on the bit value by a first mask, and an OR operation carried out on the result of this operation by a second mask. Both of the masks are programmable by virtue of a respective second operand input. An AND operation is carried out on the results from all the bits to give a single output bit. This output bit may be optionally inverted. The logical mode allows any pattern of '1's, '0's and 'don't care's in the input values to be selected as the criterion for foreground pixels and background pixels. If the second operand input to the XOR mask is a 1 then the output is the inverse of the bit value, and if the second operand input is a 0 then the output is the same as the bit value. The XOR mask therefore acts as a programmable inverter, inverting the bit value if the second operand is 1 and not inverting it if it is 0.

The bit values generated by the pre-processor 16 are output to a decision logic unit 18. The decision logic unit 18 comprises pre-programmed logic devices and a video line store, type IMSA113, to provide a single line delay so that each pixel in turn may be compared with its neighbours on the previous video line. The logic unit 18 determines the label to be applied to each pixel by applying Boolean decisions, the process for which will also be described later.

The logic unit 18 provides control signals on control lines 19 to a local labelling unit 20 comprising pre-programmed logic devices and two IMSA113 video line stores to make available the results from the previous video row. The local labelling unit 20 allocates a 16 bit label to each pixel, the label value having been determined previously by the decision logic unit 18. The labelling unit 20 also generates equivalence messages which are sent to a message queue 22. The message queue 22 comprises a First-ln-First-Out (FIFO) memory with separate read and write ports.

The message queue 22 buffers the messages and can accommodate 65 536 messages at any one time. To achieve the depth of 65 536 words, the FIFO memory is constructed from a 64 Kword SRAM (two TC551664 devices giving 32 bit width) into which messages are written, and a small, 512 element, FIFO memory (four AM4601 devices giving 32 bit width) to which messages are transferred from the SRAM and from which they can be read asynchronously by the transputer 17. A programmable logic device controls addressing of the SRAM, and transfer between the SRAM and the small FIFO.

Equivalence messages stored in the message queue 22 are passed to a transputer 17. The transputer 17 uses the equivalence message information to construct lists of local labels that correspond to the same global object. These lists are placed in a parent pointer unit 26. In discriminated mode output, but not in standard operation, the transputer output is transmitted to a global look-up table (LUT) 28. The parent pointer unit comprises two random access memories (RAMs) 30 and 32, each being a TC551664 device. The RAMs 30 and 32 are connected to an initialisation unit 34.

There are two RAMs 30 and 32 in order that one is being initialised by the initialisation unit 34 whilst the other is in use. Initialisation comprises filling the RAM with zeroes.

The transputer 17 can both send and receive information to and from the pointer unit 26 and global LUT 28.

The local labelling unit 20 also transmits the 16 bit labels for each pixel to a frame memory 36. The frame memory is a FIFO memory comprising two C1PD42280 video frame devices giving sixteen bit width. The frame memory 36 stores the local labels for the pixels in the frame to be output. The frame memory 36 is connected to a global labelling unit 38. The global labelling unit 38 is connected to the pointer unit 26 and the global LUT 28. As for the transputer 17, the global labelling unit 38 is capable of two-way communication with both the pointer unit 26 and the global LUT 28. The function of the global labelling unit 38 is to accept locally labelled pixels from the frame memory 36 and to provide each pixel with a global label. The resulting globally-labelled frame is output in the form of an 8-bit video output to a monitor 40.

The operation of the labelling system 10 will now be described in general terms a more detailed description being given later. In the description it is assumed that the pixels are eight-connected. The term "eight-connected" denotes a pixel connectivity system in which each pixel is considered to be connected to each neighbouring pixel at its edges and corners. For pixels other than those at an edge or corner of a frame, each pixel is connected to eight neighbouring pixels, i.e. one pixel at each of its four sides and four corners.

The system 10 operates on an image of 512 x 512 pixels. For eight-connected pixels the largest possible number of discrete, i.e. unconnected, objects is 65,536. This special case is a regular geometric array of one-pixel objects, with one object in every block of four pixels. However, in the system 10 the pixels in the labelled image are 8 bits wide and as a consequence only 256 labels can be supported. Usually, 256 labels are more than sufficient to label the discrete features of an image. Detection of numbers of objects greater than this indicates insufficient or inappropriate preprocessing of an image, with too much clutter or noise being allowed through preprocessing as discrete objects. In the system 10 the first 254 objects detected are given the label corresponding with the order number of their detection, i.e. the nth object detected is given label n. The labels are contiguous integers beginning with unity. If more than 254 objects are detected then all further objects detected are labelled with the value 255. The presence of more than one object labelled 255 thus indicates that an overflow has occurred and the system 10 had detected more objects than it can label.

The system 10 carries out connected component labelling in three phases as follows: A) Local Labelling In local labelling carried out by decision logic unit 18 and local labeller 20, each foreground pixel within an image is provided with a label which is based only on the local connectivity, i.e. the label on the foreground pixel is determined only by the labels of the connecting pixels. This means that for complex shapes local labelling may result in one pixel within an object having neighbouring pixels with different local labels.

B) Equivalence Processing The transputer 17 compiles a record of which local labels correspond to each global object in the image.

C) Global Labelling In normal operation, the global labeller 38 constructs contents of the global LUT 28 as it goes along. It uses the data created by the transputer 17 in the parent pointer unit 26 to determine how to relabel the pixels stored in the frame memory 36 as the pixels are being output. In discriminated output mode, the transputer 17 constructs the contents of the global LUT 28 directly. The global labeller 38 then uses these contents directly to relabel the pixels stored in the frame memory 36 as the pixels are being output.

The operation of the labelling system 10 will now be described in more detail, concentrating on the operation of the decision logic unit 18 and local labeller 20. the transputer 17 and the global labeller 38.

The decision logic unit 18 receives bit values generated by pre-processor 16 corresponding to the nature of each pixel, i.e. 1 if the pixel is a foreground pixel and O if it is a background pixel. The image is scanned in raster fashion - left-to-right across each row, starting at the top row and finishing at the bottom row.

Each pixel is processed in turn as follows. Referring to Figure 2, there is shown a five-pixel window 200 used for local labelling. For the pixel being processed, referenced 'e' in Figure 2, the window covers pixel e's nearest left neighbour pixel d, and three pixels in the previous row to which it is connected, labelled a, b. and c from left to right in pixel order. The other four neighbours of pixel e, which are not shown, cannot be processed at this time because they have not yet been scanned. At the top, left and right borders of the image, the window overlaps the edge of the image so the relevant pixels within the window are set to 0, i.e. they become background pixels.

The output for the local labelling consists of a label for each pixel. Background pixels are assigned a label of 0 and foreground, i.e. image objects. pixels are given local label values. The labels start with 1 for the first object encountered and increment by 1 each time a new object is encountered. However, it is possible that some different local labels correspond to the same object as pixels which are in different parts of the same object may be assigned different labels. This type of event is rectified later during global processing. Also, re-use of labels is not permitted as this would result in two or more probably separate objects having the same local label. Again, this would cause confusion in the global labelling phase.

The local labelling will now be described in detail. The labels assigned to pixels a, b, c, d, and e will be referred to as A, B, C, D and E, with E being the local label to be determined for the current pixel e. It should be noted that once a pixel has been assigned a local label the label cannot be changed since the label will have been stored in the frame memory 36 ready to be output to the global labelling phase.

When current pixel e has no neighbours within the window 200 it is assigned the next available label number by the local labeller 20. This occurs when the first pixel in a new object is encountered. The local labeller 20 assigns the next available number to a pixel and maintains a record of the last label assigned to a pixel. As stated previously, the largest number of possible objects in a 512 x 512 image with 8connected pixels is 65,536, or 216. The local labels are therefore stored using 16 bits and the labels are generated by a 16-bit counter (not shown) in local labeller 20.

When the pixel has only one neighbouring pixel in the window 200 then the local label assigned to it is the same as the label associated with the neighbouring pixel. When two or more foreground pixels are in the window 200 then the smallest value label is used to label pixel e.

There are 32 different possible combinations of pixels within the window 200. These are listed in Table 1 below.

Case Pixel Label Used Case Pixel Label Used Combination for E Combination for E 1 x x x zero 10 0 1 0 D x 0 1 1 2 0 0 0 next available 11 1 0 1 the smaller 0 1 free label 0 1 of A and C 3 1 0 0 A 12 0 0 1 the smaller 0 1 1 1 of C and D 4 0 1 0 B 13 1 1 1 C 0 1 0 1 5 0 0 1 C 14 1 1 0 D 0 1 1 1 6 0 0 0 D 15 0 1 1 the smaller 1 1 1 1 of C and D 7 1 1 0 B 16 1 0 1 the smaller 0 1 1 1 of C and D 8 0 1 1 1 C - 17 iii the smaller Cl 11 1 and D 9 1 0 0 D

Table 1.

In Table 1, a value of 0 indicates background pixel, 1 is a foreground pixel and X indicates the value is not important and may be either a foreground or a background pixel. In Table 1, the "Label Used for E" column indicates in each pixel configuration the value of the label to be assigned to pixel 3, where again A, B, C, D and E are the labels assigned to a, b, c, d and e respectively. The reasons for each label value for pixel e in each case are as follows: Case 1: Pixel e is background, therefore E has value 0 regardless of neighbouring pixels.

Case 2: Pixel e is foreground and has no neighbouring foreground pixels.

therefore it is the first pixel encountered in a new object. E is given the value of the next available local label (from the label counter in local labeller 20 and the counter is incremented).

Cases 3 to 6: Pixel e has only one neighbouring foreground pixel and therefore E is assigned the same value as that of the neighbouring pixel.

Case 7: Neighbouring pixels a and b are foreground. B must be less than or equal to A, because when B was originally labelled it was given the smallest label of its neighbours which included a. Therefore E is assigned the value B.

Cases 8 to 10: These follow from similar arguments to Case 7.

Case 11: Neighbouring pixels a and c are foreground. It is not possible to deduce whether A < C or C < A so an arithmetic comparison is required. E is then assigned the smaller of the two values. Case 11 typically arises when what appeared originally to be two objects eventually joined together. An example of this is shown in Figure 3. Here, object 300 has two arms 310 and 320. When previously encountered pixels in these arms 310 and 320 have appeared to be in separate objects. However, at pixel e it is evident that the arms 310 and 320 merge and consequently pixel e, and subsequently pixels in arm 330, which is labelled after further processing, are assigned the value of pixel e. A message consisting of the two labels to be merged is passed to the transputer 17 by local labeller 20 unless the value of A is equivalent to that of C, in which case no message is required.

These message are referred to as "equivalence messages".

Case 12: This follows from a similar argument to Case 11.

Case 13: The three neighbouring pixels a, b and c are foreground. From a similar argument to Case 7 it can be seen that B s A and C < B, and therefore C has the smallest value. The value of E is therefore set to be equivalent to that of C.

Case 14: This follows from a similar argument to that of Case 13.

Case 15: Pixels b, c and d are foreground. It can be deduced that D < B and C < B, but it is not possible to determine which of D and C is the smaller. This case proceeds in a similar manner to that of Case 11, with the same equivalence message being sent but with values C and D instead of values A and C respectively.

Cases 16 and 17 These follow from similar arguments to Case 15

Input Condition Label E with Equivalence Message e O None 0 aDcde A None bcde B None a+b + b)e a e C None cde D None a b c d e Next available label None a cd e MIN(A,C) If A 4 C, then merge A and C. If A = C then no message cde MIN(C,D) If C w D, then merge C and D. If C = D, then no message Table 2.

Table 2 provides a summary of the various pixel values, the corresponding appropriate value of label E and the appropriate equivalence message to be sent from local labeller 20 to transputer 17. Note in Column 1 that where a pixel has been omitted from a condition, e.g. pixel a and b in the last condition, then that label and equivalence message apply to that condition regardless of the value of the omitted pixel. MIN(X,Y) means the smaller of the arguments.

When all pixels in the image have been locally labelled then a special message is sent from the local labeller 20 to transputer 17. This special message has two functions. Firstly, it marks the end of the stream of equivalence messages and secondly, it informs the transputer 17 if any labels have been used which have not been referred to in any previous equivalence messages. The former function is necessary because the transputer 17 needs to know when the processing has finished and the latter function is used to signify those local labels for which no equivalence message has been transmitted and hence which correspond to a single object which has already been uniquely identified.

Figure 4 shows an example of the local labelling phase on an input image shown at (a). The labelled output at (b) shows five discrete objects 400, 402, 404, 406 and 408. Objects 402 and 408 are objects which contain just one local label - 5 and 12 respectively - and hence have been uniquely identified. However, objects 400, 404 and 406 each include more than one local label and hence there are a number of pixels - pixel 410 in object 400, pixels 412, 414 and 416 in object 406 and pixels 418, 420, 422 and 424 in object 404 - for which equivalence messages are sent from local labeller 20 to transputer 17. Pixel 426 is the pixel for which the special end-of-image message is sent.

Table 3 summarises the messages which are generated as the image is locally labelled.

Message Number Message 1 1-2 2 3#4 3 6-9 4 3-6 5 10-11 6 3=8 7 10~11 8 7=10 9 12#END Table 3.

In Table 3, Message Number 1 is "local label 1 is equivalent to local label 2", which is the meaning of the entry in t first local label encountered in the global object. Because of the order in which local labels are generated, the parent label of a global object is the smallest label associated with that object. The association between parent and child labels within a global object is recorded by means of tree structures. Each child label in a tree points to the parent label of the tree. The parent label points to a "null" value of zero, which is reserved for this purpose - zero is not a valid local label. since the background pixels in the locally labelled image have a value of zero. A list of parent labels is stored in the parent list memory 26. Figure 5 shows a tree structure of the local labels 1 to 12 arising from the locally labelled image of Figure 4. Before image processing the initialisation hardware 34 of Figure 1 sets all labels pointing to nulls.

Thus, as each new object is encountered, it is given a parent label until such time as another label merges with it.

When the transputer 17 receives an equivalence message from local labeller 20 the parent labels of the respective local labels in the equivalence message are located. If the two labels have the same parent label then they are in the same tree structure and hence have already been merged by a previous message, and therefore no further action is required because both labels will eventually be replaced by the value of the parent label.

However, if the two labels occur in different trees then the trees must be merged.

This is because the two labels are equivalent and therefore relate to the same object in the image. The trees are merged by making the larger of the two parent labels point to the smaller of the two. The parent of the newly merged tree remains the smallest within the tree.

The foregoing procedures are demonstrated in Figure 5. In the second row of Figure 5, labels 2 and 4 of Figure 4 are indicated by messages 1 and 2 in Table 3 to be equivalent to labels 1 and 3 respectively; i.e. 2 and 4 are child labels, whereas 1 and 3 are parents. Therefore the transputer 17 arranges for the child label pointers for labels 2 and 4 to point to the parent labels 1 and 3 respectively. Similarly, after message 3 from Table 3, label 9 points to its parent label 6. However, after message 4 from Table 3 it is apparent that parent label 6 is itself a child label of label 3, and hence in the fourth column of Figure 5 label 6 is arranged to point to label 3.

It is apparent that the tree structures shown in Figure 5 increase in depth as parent labels with associated child labels themselves become child labels to a different parent label. However, for ease of global labelling, it is preferable to restrict tree depth to a maximum of one pointer, i.e. all child labels point directly to one parent label. For global labelling to work in hard real time, it is necessary to restrict the depth to less than or equal to one pointer. It would be possible to allow the trees to grow in depth during the equivalence processing phase and then update the pointers at the end of the processing to flatten all trees to a depth of one pointer. However, it has been found in practice to be more efficient to update the trees after each equivalence message has been processed. Therefore, after two trees have been merged together the pointers of all the child labels in the tree that have been moved, i.e. the one with the larger parent label, are updated to point directly to the parent label of the overall tree. This is shown in the fifth row of Figure 5, where child label 9 is changed to point to parent label 3 having previously pointed to label 6.

The equivalence messages from the local labeller 20 are initially sent to the message queue 22, which acts as a buffer for equivalence messages before they are passed to the transputer 17. This buffering allows the transputer 17 to operate in real-time, even though each individual message is not operated in real time. This is the reason why the system 10 operates with one frame of latency instead of two or more, and distinguishes the invention from the Sumitomo Metals devices. The local labelling is deterministic and is performed in hard real time (i.e. each pixel is processed before the next one arrives). The equivalence processing is non-deterministic and is not performed in real time, but the use of a FIFO to decouple it from the local labeller and the assumption that the occurrence of messages with a suitably processed image is sufficiently sparse, means that the equivalence processing can on average keep up with the local labelling. Table 4 shows the final state of the pointers in Figure 5.

Local Label Parent Pointer Local Label Parent Pointer 1 NULL 7 NULL 2 1 8 3 3 NULL 9 3 4 3 10 7 5 NULL 11 7 6 3 12 NULL

Table 4.

The global labelling phase will now be described. The global labelling phase is carried out by the global labeller 38 which uses the locally labelled image and the list of parent pointers generated in the local labelling phase and the equivalence processing phase respectively. The purpose of the global labelling phase is to label the image such that each object in the image is labelled with one global image.

The locally labelled image is scanned in raster order. If the pixel value is zero then the pixel is background and a value of zero is output. It the pixel value is non-zero then it is a foreground pixel in an object. The local labeller is used to address the list of parent labels, stored in parent list memory 26. If the parent pointer is null then the pixel must be the first pixel in a new object. It is therefore labelled with the next available, unused global label. These labels are issued by a counter, which starts at 1 and increments each time a label is issued. The counter counts up to 255 and then stops so all new object encountered after the 254th are labelled with 255. The global LUT 28 is created during the global labelling phase which gives the global label assigned to each parent label. As the global label is issued by the counter, it is copied into the LUT at the appropriate location for the parent label. Finally, the value of the parent label in the list of parent pointers is changed from null to that of the parent label itself, i.e. the parent label points to itself. The reason for this is that when another pixel with the same local label is encountered the parent value is no longer null and a new global label is not issued.

If a parent pointer is non-zero, which is the case for a child label or a parent label that has already been accessed at least once, then the pointer gives the value of the parent label. The LUT 28 entry corresponding to this label is read, and this gives the global label that was originally assigned to the parent label when it was first accessed. This is the value used to label the output pixel.

The global labelling works for two reasons. Firstly, the first pixel to be encountered in any object will have the parent label for that object, and therefore the LUT 28 entry is bound to be initialised (i.e. written) before it is used (i.e. read). This is true because the local labelling issued labels in increasing numerical order so the first pixel in a new object will have the smallest label within the object and therefore when the transputer 17 merged two objects it always retained the smallest parent label as the parent label of the new object. Secondly, the parent trees are no more than one branch deep, so that exactly one read access is required from the list of parent pointers in order to determine the parent label. If the parent trees had been allowed to grow to an arbitrary depth then it would not have been possible to determine the parent label in a fixed time. This would prevent the global labelling from operating in real time.

The real time operation allows for a latency of only one frame, providing the transputer 17 can complete its processing during the first frame.

Whilst the local labelling phase is proceeding it is possible to generate simple parameters for the objects encountered in the image. The local labeller 20 may be arranged to provide information to one or more statistics modules 42 for this purpose.

For example, the number of pixels with each local label may be counted and this, when combined by the transputer 17 with the data from the equivalence processing phase, will give the size of each global object in the image. Other parameters which may easily be calculated are centroid co-ordinates for each object, and the coordinates of a bounding box.

The transputer 17 may also be used to construct an output LUT directly when in the discriminated output mode. If the processor 17 has access to the statistical parameters that have been calculated during the local labelling phase then while it is constructing the LUT it can discriminate against objects with specific characteristics.

For example, all objects in the image which are smaller than a certain threshold size can be removed from the image. This is a common requirement in image processing systems, where small objects may represent noise.

Claims (7)

Claims

1. An image labelling system for labelling objects in an image comprising a plurality of pixels, the objects comprising at least one foreground pixel, wherein the system comprises: (i) means for identifying each pixel of an image as either a foreground pixel or a background pixel; (ii) local labelling means for assigning a local label to foreground pixels such that connected foreground pixels may receive the same local label and for generating equivalence messages when a pixel within an object has neighbouring pixels with differing local labels; (iii) global labelling means for assigning global labels to objects in the image such that objects comprising a plurality of pixels with differing local labels are assigned a single global label; and (iv) equivalence processing means for receiving the equivalence messages from the local labelling means and for determining which locally labelled pixels correspond to the same object; and wherein the local labelling means and the equivalence processing means are arranged such that the operation of the local labelling means is not delayed by the operation of the equivalence processing means.

2. A system according to Claim 1 wherein the equivalence processing means includes means for associating with each object comprising more than one local label a parent local label and one or more child local labels and wherein the equivalence processing means also includes parent label storing means for storing a list of the parent local labels and an indication as to which parent local label each child local label is related.

3. A system according to Claim 2 wherein the equivalence processing means is arranged to link the child local labels to the parent local labels such that each child local label is directly linked to a parent local label.

4. A system according to any preceding claim wherein the equivalence processing means includes processing means for determining which locally labelled pixels correspond to the same object and a buffer memory for receiving the equivalence messages from the local labelling means and for storing the messages until the processing means is ready to receive them.

5. A system according to Claim 4 wherein the system includes global storage means for storing a list of which parent local label corresponds to which global label and wherein the global labelling means is arranged to receive data from and send data to the global storage means.

6. A system according to Claim 4 wherein the system includes global storage means for storing a list of which parent local label corresponds to which global label and wherein the equivalence processing means is arranged to send data to and the global labelling means is arranged to receive data from the global storage means.

7. A system according to any preceding claim wherein the system includes means for receiving data from the local labelling means and for generating parametrical information relating to each of the objects in the image.