EP0873654B1 - Bildsegmentierung - Google Patents

Bildsegmentierung Download PDF

Info

Publication number
EP0873654B1
EP0873654B1 EP97944071A EP97944071A EP0873654B1 EP 0873654 B1 EP0873654 B1 EP 0873654B1 EP 97944071 A EP97944071 A EP 97944071A EP 97944071 A EP97944071 A EP 97944071A EP 0873654 B1 EP0873654 B1 EP 0873654B1
Authority
EP
European Patent Office
Prior art keywords
segmentation
portions
predicted
regions
pixel data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP97944071A
Other languages
English (en)
French (fr)
Other versions
EP0873654A1 (de
Inventor
Timothy Ian Paterson Trew
Richard David Gallery
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Publication of EP0873654A1 publication Critical patent/EP0873654A1/de
Application granted granted Critical
Publication of EP0873654B1 publication Critical patent/EP0873654B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/215—Motion-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T9/00—Image coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding

Definitions

  • the present invention relates to image segmentation with improved temporal consistency, and to image processing including steps dependent on segmentation.
  • An image is conventionally described by defining the attributes of each pixel of the image, in particular the pixel colour.
  • the attributes defining each pixel usually consist of the grey scale value of the pixel whereas in a colour image a plurality of colour component values need to be defined per pixel.
  • the invention is not limited to these conventional images and attributes, however, and applies to any spatial attribute which can conveniently be represented in the form of a pixel array in two or more dimensions.
  • the concept of time and temporal consistency should be understood to include analogous dimensions, such as when segmentation of tomographic image "slices" is performed for a sequence of images over a third spatial dimension, not necessarily in a time sequence.
  • Image segmentation is a known technique which groups pixels into regions, each region containing only pixels having similar attributes.
  • the technique has many applications particularly in the field of image coding (compression).
  • Image coding schemes using image segmentation are particularly suitable for low data rate transmission of image motion. Typical data rates may be as low as 64 kbits/s. They are, therefore suitable for applications such as video-phones which require the transmission of real time video information down the narrow bandwidth of a telephone line. Even if the segmentation is not itself encoded, the segmentation can be useful, for example, to concentrate the available bandwidth on the 'important' parts of the image, such as the face of the person speaking.
  • An image coding scheme which uses image segmentation explicitly is region and texture coding as described in published patent applications EP-A-0437002 (PHB 33610) and EP-A-0454234 (PHB 33626).
  • Japanese Patent Application 08149461A to Sony Corporation discloses a method to extract the three-dimensional shape information on each segment constituting a series of moving images and to code the series of moving images with a high compressibility by using the information.
  • segmentation schemes are used for the coding of a series of frames in a motion picture, the visual artifacts which result from segmentation carried out on individual frames will change with each frame and may produce a subjectively very displeasing image sequence representation. It is therefore desirable that the segmentation be temporally consistent.
  • image segmentation is not limited to use in image coding and can be used generally in diverse image processing applications such as image enhancement, object tracking, extraction of 3-D geometry from images, computer-aided animation and colourisation.
  • a problem with this approach is that it relies upon a satisfactory division of the image into its constituent objects. This in turn relies either upon prior knowledge of the objects likely to be present in the image or is derived from complex processing of plural subsequent image frames. If there is only minimal knowledge of such objects, the modelling becomes very difficult and the splitting of the image into such objects cannot be satisfactorily achieved. Failure to segment the image satisfactorily tends to produce subjectively very displeasing results. For example, in one experimental video-phone application it has even occurred that a nose grows from the forehead of the transmitted face. As previously stated, this approach requires a complicated and extensive database of object models and additionally the matching of such models to the objects in the image may require excessive computation. Thus this approach is not presently a reliable technique for general image coding.
  • the present invention aims to provide an approach to temporally consistent segmentation that does not require the specific modelling of objects in the image.
  • a method of operating an image processing apparatus to produce a consistent segmentation into regions of a sequence of related image frames such that like groups of pixels belong to like regions in succeeding frames comprising for each frame of interest the steps of:
  • an image processing apparatus for producing a consistent segmentation into regions of a sequence of related image frames such that like groups of pixels belong to like regions in succeeding frames, the apparatus comprising:
  • a computer-usable medium storing computer-readable instructions for causing a process in a computer graphic apparatus to process signals defining current frame pixel data for a frame of interest, reference frame pixel data and a reference frame segmentation, the instructions comprising instructions for causing the processor to:
  • the temporal consistency imposed by use of the invention can also reduce the computation involved when simplistic assumptions relating to the motion in the image sequence are made, by providing a mechanism to detect and correct errors when such assumptions are invalid.
  • the present invention enables the provision of a consistent segmentation for a series of related pictures, for example to produce temporal consistency to the segmentation of a motion picture sequence containing moving objects.
  • a method of segmentation comprises some or all of the following steps:
  • step (f) various heuristics may be applied to reduce the number of identified pixels, by allocating them to the same regions as neighbouring pixels, and/or to designate further pixels for consideration in step (f).
  • the segmentation performed in accordance with the present invention provides improved temporal consistency and therefore, although artifacts will still be present in the image segmentation, these will be consistent from frame to frame and hence their consequences will be less displeasing to a viewer.
  • an embodiment of a method according to the present invention assumes that all interframe motion can be characterised as two-dimensional planar motion so that a conventional two-dimensional planar motion estimator, such as a block matching estimator, may be used to calculate the motion vectors for each pixel of the image. Not only is this assumption not generally valid for real-life sequences but, even if the interframe motion were to consist exclusively of two-dimensional planar motion, as the motion vectors are only calculated by the block, it is extremely unlikely that accurate motion vectors will be calculated for all the individual pixels in each block. In order to detect inaccurate motion vectors the embodiment calculates a displaced frame difference (DFD) of each pixel.
  • DFD displaced frame difference
  • the DFD represents per pixel the error between a frame and the estimate of that frame provided by the motion vectors based on the neighbouring frames. An excessive DFD may thus indicate that the motion vectors for a particular pixel are invalid. The DFD is then used to identify where the segmentation predicted by the motion vectors requires correction.
  • Figure 1 provides an overview of the image segmentation process for a series of frames of a motion picture.
  • the image is monochrome, the intensity of each pixel being defined by a single grey scale value.
  • the invention is not, however, limited to monochrome processing and the variations necessary to allow for a coloured image or any form of pixel array will be apparent to a person skilled in the art.
  • process steps to be performed are designated by oval boxes and the major data structures involved by rectangular boxes.
  • frame F1 is the first frame of the series, it is subjected to an initial segmentation process 100 to provide segmentation data S1 in a manner to be described later.
  • motion vectors M are calculated in a conventional manner described below more fully with reference to Figure 3 and step 21, from the grey scale frame data F1 and the new grey scale frame data F0. Again, a number of different techniques are known for the calculation of such vectors but in the present embodiment motion is modelled as two dimensional translations of small blocks of pixels.
  • This prediction S0 1 will, however, contain invalid entries for a number of pixels due, in part, to whatever approximations and assumptions were made in the motion detection process, for example the blocking of pixels. Errors may also be introduced by noise, either external or within the image processing system and by "uncovered background" that cannot be predicted.
  • the embodiment therefore identifies pixels for which the segmentation may be invalid (the criteria for invalidity will be described later) and applies various optional processes in stages to such pixels in order to attempt to allocate them to existing regions. Any pixels which are not successfully allocated are then allocated to new regions created by carrying out a segmentation process from scratch on the unallocated pixels.
  • the first category of pixels are those which lie on the borders of the predicted regions. Such pixels have been identified by the inventors as particularly susceptible to noise. In addition, because motion vectors are conventionally calculated for a block of pixels rather than individual pixels, errors will tend to be introduced around the borders of regions (objects) where a block contains pixels properly belonging to different objects.
  • the second category are those pixels which are judged as having failed motion compensation.
  • the motion vectors and pixel data of the image frames are criteria used to determine failure of motion compensation, as will be discussed in fuller detail later.
  • Process 104 broadly identifies pixels which lie at the borders of the regions to which they are assigned in the predicted segmentation (category 1 pixels). As will be described in more detail below, the embodiment then attempts to reallocate such pixels to more suitable regions according to the pixel values and calculated region statistics.
  • Process 106 attempts to allocate to suitable regions those pixels for which the motion vector of the block appears not to indicate the motion of the pixel correctly (category 2 pixels), these pixels having been excluded from consideration in process 104. This exploits in particular the inference that for those pixels where the pixel value has not been correctly predicted by the motion vector of the block to which it belongs, then the pixel's region will also not have been correctly predicted in the predicted segmentation.
  • Those pixels which have failed motion compensation (category 2) but which lie at or near the boundary of one or more existing regions may then be allocated to one of those regions. As described more fully below, this is achieved in the present embodiment by considering the gradient of the pixel values at each such pixel.
  • the new segmentation S0 is taken as the initial segmentation (S1) and the process is repeated.
  • FIGS 2A, 2B and 2C show in more detail the image segmentation in the embodiment.
  • grey scale data for frame F1 is subjected to initial segmentation process 11 in any suitable manner, depending on the image content and the purpose of segmentation.
  • An example of such segmentation will be referred to as the Brice-Fennema technique, and is described in "Scene analysis using regions” (Artificial Intelligence 1 (1970) 205-226).
  • a pixel array is created and a value is entered for each pixel to identify the region of which it is a member.
  • This representation of the segmentation is used in preference to the chain of "boundary segments" described in the above mentioned paper, as it simplifies the succeeding steps of the present embodiment. However, such a representation is not essential and the chain representation could instead be used.
  • the chain representation is used during the initial segmentation and converted to the pixel array form for updating the segmentation from frame to frame.
  • the result of segmentation process 11 is segment data S1.
  • various region statistics are calculated for each region of S1.
  • the statistics calculated are the number of pixels in each region, the mean grey scale value of each region and the standard deviation of each region.
  • the mean and standard deviation being stored explicitly for each region, the sum and the sum of the squares of the pixel grey scale values are stored.
  • Small regions are next "eliminated" in step 15.
  • those regions smaller than 5 pixels in size are removed although this is an empirically determined value and should not be construed as limiting. In practice, this typically leaves 1000 regions in the frame for an image of 360 by 280 pixels. Eliminating small regions reduces the subsequent processing for the present frame, and also counteracts the tendency for the number of regions to grow during the sequence of images. The elimination could, alternatively, be performed at a later stage in the method but its performance at this stage is preferable as it eliminates the subsequent processing of small regions which may not prove of any significance. Alternatively, by careful choice of the segmentation process and the criteria applied in such a process it may be possible to avoid any small regions being created.
  • the process of elimination is implemented by merging each small region with the neighbouring region that has a mean grey scale value closest to that of the region to be removed.
  • the segmentation array S1 becomes overwritten with the new region structure S1 1 .
  • the statistical record STATS for each region is updated in step 17.
  • the statistical records are merely updated by considering which pixels have been removed or been joined to each region. It will be appreciated that in practice this step can be integrated with the elimination of small regions (step 15). It will also be appreciated that it is possible completely to recalculate the statistics for each region after the elimination step 15 is complete. This may, for example, be preferable when the statistics are calculated in hardware. As a further alternative, because the neighbouring region will typically be much larger than the eliminated small region, the statistics of the neighbouring region can be used without any alteration as a reasonable approximation for the new region statistics. In other words, step 17 is optional.
  • FIGS 2B and 2C illustrate the various processes carried out on segmented frame S1 1 after the initial segmentation S1 1 is established to provide output segment data S0 for the current frame F0, as will now be described.
  • step 19 the motion vectors M are used to motion compensate the segment data S1 for frame F1, on a pixel by pixel basis in order to give a first prediction of segment data for frame F0.
  • This prediction is designated S0 1 .
  • Different methods of motion compensation are known in the art. In the present embodiment, it is assumed that the motion vectors M are provided one for each block of the current frame F0, and will indicate the relative location of pixels in a segmentation data S1 whose region ID can be copied to the pixels of the block presently under consideration in array S0 1 , even if the block matching performed by the motion estimator yields a very poor match.
  • Step 21 clears the individual pixels of S0 1 which are flagged as invalid by a first array of flags U10.
  • Figure 3 shows the determination of the flags U10 which are related to the displaced frame difference (DFD) of each pixel.
  • Data F1 and F0 is input into motion detector 1 in order to calculate motion vectors M.
  • motion detectors are well known in the art, being conventionally used in image coding for high definition TV, and for lower bit rate image coding schemes such as the well known MPEG and H.261 standards. Examples of such motion detectors are described in "Motion Estimation and Compensation” by Gerard de Haan (1992), ISBN 90-74445-01-2 and “Advances in Picture Coding” by Hans Musmann, Peter Pirsch and Hans Joachim Grallert (Proceedings of the IEEE, Vol 73, No. 4, April 1995).
  • the motion detector used in the present embodiment operates only in respect of two-dimensional planar movement and calculates the translation of blocks of 8 x 8 pixels in the x and y directions.
  • a more complex motion detector could be employed capable of modelling transformations more complex than simple translation and that the blocks could be of various sizes or even a single pixel.
  • a further alternative is the calculation of motion vectors by regions.
  • the simple method of calculating such motion vectors adopted in the present embodiment is to consider each block of pixels in the current image frame F0 in turn, searching through previous frame F1, to find a block of pixels which best matches i.e. to see where the present block of pixels is most likely to have come from. Searching in frame F1 is conducted typically in an outward spiral from the block's location in F0.
  • Such backward motion vector calculation has the advantage over the alternative of forward motion vector calculation that a vector is produced for each and every block of F0.
  • forwards motion vectors could be used, in particular where these are available already in the apparatus.
  • the lack of a motion vector for certain pixels in F0 is not fatal as the embodiment could be adapted to set flag U10 (motion vector invalid) directly for such pixels.
  • the motion detector 1 will provide a motion vector for each block of pixels in F0.
  • Frame F1 is then subjected to a process of motion compensation 3 using the derived motion vectors M.
  • the motion vectors are applied to the frame F1 to give a prediction of the frame F0.
  • This prediction is designated F0 1 .
  • each pixel does not have its own exclusive motion vector it will be apparent that the motion vector for the block containing that pixel will be used.
  • Motion compensation processes are again well known in the field and need only be matched to the type of motion detection process used and the consequent form and meaning of the motion vectors available.
  • the segmentation prediction S0 1 is produced (step 19) from the segmentation S1 using the same motion vectors in the same manner.
  • F0 and F0' are then subjected to pixel difference calculating step 5 which detects, for corresponding pixels, the difference in the grey scale values of each pixel between the two arrays i.e. the difference between the pixel's actual value and its predicted value.
  • each pixel is considered in turn and the pixel difference calculation provides an absolute value (magnitude) of pixel difference.
  • a pixel array of displaced frame difference (DFD) is thus produced, designated DFD in the figure.
  • the DFD values may be available already as a by-product of the block matching process in step 1, depending on the implementation.
  • the value of DFD is tested for each pixel to determine whether it exceeds a threshold T.
  • the data provided by the thresholding is stored in flag array designated U10 in the figure, and is indicative of whether the motion vector for the block containing the pixel has failed to predict the value of that individual pixel.
  • a fixed threshold is used for all the values of DFD, but this is not essential and instead the value of the threshold could be varied.
  • processing could be carried out to detect objects (such as foreground and background) and different thresholds applied to these different objects.
  • a flag as part of the motion vector calculation step 1 it is possible to set a flag as part of the motion vector calculation step 1 to indicate that such a good match for the block has been found, that the motion vector will be valid and thus for that block of pixels it is unnecessary to calculate the DFD.
  • a flag such as the U10 flag directly during motion vector calculation to indicate that such a bad match had been made that the motion vector is almost certainly invalid.
  • step 21 clears the pixels flagged by U10 in the predicted segmentation S0 1 step. This is achieved by setting a "null" region ID in the array S0 for each pixel to be cleared. This step realises the inference mentioned in the introduction, namely that, if a pixel's value has been poorly predicted, then its predicted region ID is probably wrong too.
  • the output from this process is designated segmentation S0 2 .
  • This second flag array is labelled U20 and is calculated in step 23.
  • the flags U20 indicate which of those pixels contain region IDs that are nominally valid (i.e. U10 is not set) but which are poorly connected to their regions.
  • the criterion imposed to judge such poorly connected pixels is whether a pixel is fully 8-way connected to pixels belonging to the same region.
  • a fully 8-way connected pixel has two horizontal, two vertical and four diagonal neighbours, all with the same region ID (as stored in S0 2 ). If the pixel is not fully 8-way connected, to some extent the pixel lies on the border of a region. The pixel array of flags U20 is thus created.
  • Pixels on borders are then assigned to a suitable region if possible by border assignment step 25.
  • This process uses the U20 flag in addition to the previously created statistical data STATS (see step 17, Figure 2A) and F0 data.
  • Certain artifacts are commonly created at region boundaries by noise within the system, and also "external" noise within the source images. For example, adverse effects are often apparent when an object which is illuminated by a 50Hz source is viewed by a 60Hz camera.
  • Step 25 assists in the elimination of these artifacts by reassigning the pixels at region boundaries to the most appropriate regions, which are not necessarily the regions to which they were previously assigned.
  • step 25 compares the grey scale of the pixel (x), the mean grey scale of the region ( ⁇ ) and the standard deviation of the region ( ⁇ ). A neighbouring region is judged suitable to have the pixel of interest assigned to it if the following inequality is satisfied: x - ⁇ ⁇ 1.5 ⁇ ⁇
  • the pixel is assigned to whichever region minimises the deviation from the mean.
  • the appropriate region ID is entered into the segmentation, overwriting the array S0 2 and the U20 flag is cleared.
  • the updated segmentation will be referred to as S0 3 and the updated U20 flag array will be referred to as U20 1 .
  • Step 27 "confirms" such pixels as allocated to the same region that to which it was allocated in the predicted segmentation.
  • Step 31 attempts to allocate pixels which have failed motion compensation (those flagged in array U10) by considering whether the pixels lie on an edge feature in the image and can be allocated to an appropriate neighbouring region.
  • the most appropriate region is determined by consideration of the grey scale gradient at the pixel as follows.
  • Figures 4A and 4B illustrate the gradient calculation step 9 in the present embodiment.
  • the gradient calculation used is well known and involves the application of two masks each 3 x 3 pixels in size through the array F0 to give a pixel array of gradients G, each comprising an x component G x and a y component G y .
  • These masks are known as "Sobel operators" and are shown in Figure 4B.
  • These operators G x and G y provide a value representative of the grey scale gradient at the pixel corresponding to the central point of the mask. It will be appreciated that an alternative gradient calculation could, instead, be used.
  • step 3103 it is considered whether the pixel has been flagged by flag U10. Only U10 flagged pixels are considered in step 31.
  • step 3105 it is considered whether the magnitude of the gradient of the pixel in question exceeds a threshold. Where the range of possible grey scale values is 0 to 255, a threshold of 15 has been determined empirically by the inventors as providing a suitable indication of whether an edge feature is present (a line of contrast between two colours). It will be apparent that, if a different range of grey scales is used or if a colour image is processed then the calculation and/or the threshold magnitude of the gradient would be adjusted accordingly. Where the magnitude does not exceed 15, the pixel is not considered further in Figure 5 and the process moves to the next pixel.
  • step 3107 it is considered whether one or both of the adjacent pixels in the direction of the identified edge are allocated to a region. If neither pixel is allocated then the process moves to the next pixel.
  • step 3109 If one (but not both) of the adjacent pixels is allocated to a region it is considered, at step 3109, whether the magnitude of the gradient at the current pixel is greater than the magnitude of the gradient at the adjacent pixel which is already allocated to a region. If such a relationship exists then the pixel is allocated to the same region as that of the adjacent pixel (step 3111) and the process moves to the next pixel. If the test at 3111 is not satisfied, the process moves to the next pixel without assigning the pixel to the region.
  • step 3107 it may be determined that both adjoining pixels are already allocated to regions. In this case the process branches to step 3113.
  • step 3113 it is considered whether the magnitude of the gradient of one (but not both) of the adjacent pixels is less than the magnitude of the gradient of the current pixel (similar test to step 3109). If one adjacent pixel does not satisfy this relationship, then the other adjacent pixel which does have a lower gradient has its region ID copied to the current pixel in step 3115. In other words, the edge is apparently getting steeper in the direction of the higher gradient and hence the pixel is more likely to belong to the region of the pixel with the lower gradient.
  • step 3113 it may be determined that either both of the neighbouring pixels or neither of the neighbouring pixels have a lower gradient.
  • step 3117 allocates the pixel to the same region as that of the neighbouring pixel with the lower gradient.
  • the edge is apparently getting steeper in both directions, however, it is less steep in the direction of the neighbouring pixel with the lower gradient and so the pixel of interest belongs to the region allocated to that pixel.
  • the pixel lies on an edge and therefore, to some extent, the region to which it is allocated is arbitrary.
  • the pixel with the lower gradient is likely to be further from the notional edge and so the pixel of interest is allocated to the same region as that pixel.
  • Figure 6A is a graph of grey scale values around a typical edge feature between white pixels on the left side of the graph and black pixels on the right side of the graph.
  • Figure 6B represents the gradient of the edge depicted in Figure 6A.
  • a notional boundary between regions is aligned with the peak of the gradient curve.
  • the positions of four neighbouring pixels are labelled P1, P2, P3 and P4.
  • a first case will be illustrated where the pixel of interest is P2.
  • region ID null
  • the gradient at P2 is greater than that at P1.
  • the allocation process will be conducting the test depicted at step 3109 of Figure 5. This test will be satisfied and P2 will thus be allocated to the same region as P1.
  • P3 is allocated to a region but P1 is not
  • P2 will not be allocated to the same region as P3 because P3 has a gradient greater than at P2. Because the gradient is greater at P3 than P2, it is assumed that P2 is not on the same side of the peak as P3, and is hence on the other side of the notional boundary between regions.
  • P3 is assigned to the same region as that of P2 (step 3115) as this is more likely on the same side of the edge.
  • the step 31 is considered optional, but can be effective to compensate for slight shifts in edge features that will produce large DFD values, and thereby to reduce further the number of unallocated pixels to be processed at subsequent stages, by means of simple heuristic tests. These particular heuristics are of course only examples of the criteria that may be employed in the detection and allocation of pixels near edge features. The step could in principle be iterated more than once, to allocate further pixels as the regions "grow" to absorb the edge features.
  • step 33 the two flag arrays U10 1 and U20 2 are combined into a single flag array U0 which flags those pixels which still have not been satisfactorily allocated to a region.
  • Such combination can be achieved by a simple logical OR, and the storage space for either U10 or U20 can be overwritten with the new array U0.
  • allocation step 35 the connectivity of U0 flagged (unallocated) pixels to other unallocated pixels is considered. Those pixels which have low 8-way connectivity are allocated to adjacent regions as follows.
  • step 35 each pixel flagged by U0 is checked to see whether its 8-way connectivity to other U0 flagged pixels is greater than 2. If not, then the pixel is allocated to whichever of the neighbouring regions has a mean grey scale closest to that of the pixel according to the stored region statistics. The U0 flag is thereafter cleared for this pixel. The updated flag is referred to as U0 1 .
  • the justification is that poorly connected pixels, isolated from other unallocated regions, are unlikely to form by themselves regions of any great interest. In the present embodiment, step 35 therefore helps to inhibit the generation of small regions, which would only be eliminated at a later stage. Step 35 also eliminates certain long thin strings of unallocated pixels that can arise along region boundaries. Step 35 is also considered optional, and the heuristics applied can be varied. In particular, values other than 2 could be used for the connectivity threshold. A value of 3 will have a stronger effect on elongate regions, a value of 1 will tend to eliminate only very isolated pixels.
  • step 35 is iterated twice in the present embodiment, as the first pass may result in new poorly connected pixels.
  • the updated segmentation after the second iteration is designated S0 6 .
  • step 39 This process allows removal of a number of unnecessary regions.
  • the merged segmentation is designated SU0 1 .
  • step 41 the newly created regions of SU0 1 are merged with the existing regions of S0 6 .
  • the merging criteria may be the same as the earlier merging step 39 or different. In the embodiment the same merging criteria are used. Again merging reduces the number of unnecessary regions by combining adjacent regions of similar grey scale statistics.
  • the final segmentation for frame F0 has been obtained, designated S0.
  • S0 is then used in the image coding or other processing as appropriate to the application.
  • the updated segmentation S0 is then used as the initial segmentation for a next frame, and the whole process of Figures 2A-2C is repeated as necessary to obtain a temporally consistent segmentation throughout the motion picture sequence.
  • Figure 7 shows an image encoding apparatus incorporating the temporally consistent segmentation process described above.
  • the encoder employs region and texture encoding, more details of which can be found in EP-A-0454234, mentioned above. A brief description only will be given here.
  • a luminance picture signal Y is applied to a segmentation device 72, which implements the process described above, and a modal filter 73 uses the segmentation to produce a region signal. Both the picture signal and the region signal are applied to a subtractor circuit 75 to produce a texture signal which is encoded in encoder 77.
  • the region signal is applied to an edge mapping device 79 to produce a region list signal and an edge map of the original image.
  • the edge map is subjected in an element prediction device 710 to a template to produce a prediction from a look-up table in a memory 712 regarding the value of an element in a fixed position adjacent the template.
  • a prediction error signal indicates no error while if an error exists either the prediction error signal conveys the actual value or where it may be found in a look-up table.
  • the prediction error signal is encoded in a second encoder 714.
  • the region list signal at 716, the texture signal at 78 and the error signal at 715 can be transmitted at low bandwidth to a decoder which reconstructs approximately the original image.
  • the segmentation device 72 supplies the modal filter 73 with a segmentation which is temporally consistent from frame to frame.
  • Motion information is not used to compress the region signal or the texture signal in the present example. Nevertheless, an improved subjective image quality is obtained when the motion picture sequence is decoded from the region and texture information, for a given bandwidth, due to the temporal consistency which has been imposed on the segmentation.
  • the range of possible uses of the segmentation in image processing is not at all limited to the examples detailed above.
  • the segmentation may be explicitly coded, as in Region and Texture Coding, or it may be used as a "hint" for the coding of pixel data, particularly in schemes such as H.261 and MPEG with a variable bit rate per pixel or per block.
  • the tracking of specific objects or classes of image features can also be assisted, for image enhancement or processing generally. For example, it may be desired automatically to increase contrast in regions of a specific colour in microscopic or radiographic motion sequences, or to perform some further processing automatically only on such regions. Further diverse applications including machine vision, and digital film and video special effects such as colourisation will be apparent to the skilled reader.
  • the processes described can be extended readily to cope with multicolour images, in a variety of ways.
  • the pixel values processed by the segmentation routine may be pre-processed to enhance certain attributes.
  • a particular colour mapping might produce a modified "grey scale" image wherein skin tones are highlighted, and this can be used instead of the original colour values at one or more stages of the process.
  • Such techniques are generally peripheral to the inventive concepts disclosed herein, but can be effective in extending greatly the quality and utility of the segmentation results.
  • the present processes need not be applied to every successive frame in the source motion picture sequence.
  • intraframe coding, interframe prediction and bidirectional prediction are applied to different subsets of the source frames (I pictures, P pictures and B pictures).
  • temporally consistent updating of segmentation need not be applied to every frame, and/or can be applied differently to different subsets of frames.
  • the invention may be embodied in a computer such as an IBM compatible personal computer (PC) or the like.
  • the computer will typically comprise fixed and removal storage media such as hard and floppy disc drives, or a CD-ROM.
  • the sequence of instructions necessary for causing the computer to function in accordance with the invention may be stored on the fixed or removable storage media, for subsequent reading into the internal memory for controlling the central processing unit of the computer.
  • the program instructions may alternatively be delivered by communication signals from a remote storage device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Claims (39)

  1. Verfahren zum Betreiben einer Bildverarbeitungsanordnung zum Erzeugen einer einheitlichen Segmentierung in Gebiete einer Sequenz relatierter Bildframes, so dass gleiche Gruppen von Pixeln zu gleichen Gebieten in aufeinander folgenden Frames gehören, wobei das Verfahren für jedes interessante Frame die nachfolgenden Verfahrensschritte umfasst:
    a) das Empfangen aktueller Framepixeldaten (FO) für das betreffende Frame, Bezugsframepixeldaten (F1) und eine Bezugsframesegmentierung (S 1);
    b) das Berechnen (1) von Bewegungsschätzungsinformation (M) aus den aktuellen Framepixeldaten und den Bezugsframepixeldaten um räumliche Transformationen darzustellen, die zwischen den genannten Frames auftreten;
    c) das Verwenden (102) der genannten Bewegungsschätzungsinformations- und der Bezugsframesegmentierungsdaten zum Erhalten einer vorbestimmten aktuellen Framesegmentierung (SO1);
    d) das Verwenden (3) der genannten Bewegungsschätzungsinformation und der Bezugsframepixeldaten zum Erhalten vorhergesagter aktueller Framepixeldaten (FO1);
    e) das Identifizieren (21) nicht einwandfrei vorhergesagter Teile der vorhergesagten aktuellen Framesegmentierung durch einen Vergleich (5) der empfangenen aktuellen Framepixeldaten und der vorhergesagten aktuellen Framepixeldaten; und
    f) das Modifizieren (104, 106, 108) der vorhergesagten aktuellen Framesegmentierung für die identifizierten Teile durch Zuordnung wenigstens einiger Pixel derartiger Teile zu bestehenden Gebieten.
  2. Verfahren nach Anspruch 1, wobei der Schritt (f) das Konstruieren (37) und das Einfügen (41) einer neuen Segmentierung der identifizierten nicht einwandfrei vorhergesagten Teile in die vorhergesagte aktuelle Framesegmentierung umfasst.
  3. Verfahren nach Anspruch 2, wobei der Schritt (f) vorbereitende Schritte umfasst, und zwar der Zuordnung (25, 27) von Teilen zu denselben Gebieten wie Nachbarteile, die in dem Schritt (e) nicht identifiziert wurden, vor der Konstruktion der genannten neuen Segmentierung.
  4. Verfahren nach Anspruch 2 oder 3, wobei die genannte neue Segmentierung einem Vermischungsprozess (39) ausgesetzt wird, und zwar zum Reduzieren der Anzahl neu geschaffener Gebiete vor der Einfügung in die vorhergesagte Segmentierung.
  5. Verfahren nach Anspruch 2, 3 oder 4, wobei die Segmentierung einem Vermischungsprozess ausgesetzt wird, und zwar vor der Einfügung der neuen Segmentierung für die identifizierten Teile, zum Reduzieren der Anzahl Gebiete in der erhaltenen Segmentierung.
  6. Verfahren nach einem der vorstehenden Ansprüche, wobei der Schritt (e) weiterhin Folgendes umfasst: das Verwenden der vorhergesagten Segmentierung zum Zuordnen weiterer Teile der vorhergesagten Segmentierung als nicht einwandfrei vorhergesagt.
  7. Verfahren nach Anspruch 6, wobei die genannten weiteren Teile, die als nicht einwandfrei vorhergesagt bezeichnet wurden, diejenigen Teile sind, die an den Grenzen der Gebiete in der vorhergesagten Segmentierung liegen.
  8. Verfahren nach einem der vorstehenden Ansprüche, wobei der genannte Modifizierungsschritt (f) einen Schritt umfasst zum Zuordnen nicht einwandfrei vorhergesagter Teile zu bestehenden Gebieten, und zwar abhängig von den zugeordneten Gebieten benachbarter Teile.
  9. Verfahren nach Anspruch 8, wobei der Schritt (f) das Zuordnen wenigstens einiger Teile der nicht einwandfrei vorhergesagten Teile denselben gebieten zugeordnet werden, wie die einwandfrei vorhergesagten Nachbarteile, und zwar in Abhängigkeit von Pixeldatengradienten (G) in dem aktuellen Frame.
  10. Verfahren nach Anspruch 9, wobei nicht einwandfrei vorhergesagte Teile mit Gradienten über einer Schwelle denselben Gebieten zugeordnet werden, wie Nachbarteile in Abhängigkeit von den Zuordnungen und Gradienten der genannten Nachbarteile.
  11. Verfahren nach Anspruch 8, 9 oder 10, wobei der Schritt (f) das Zuordnen wenigstens einiger Teile der nicht einwandfrei vorhergesagten Teile denselben Gebieten wie die einwandfrei vorhergesagten Teile zugeordnet werden, und zwar in Abhängigkeit von Statistiken der Pixeldaten für diese Gebiete.
  12. Verfahren nach einem der vorstehenden Ansprüche, wobei in den Schritten (e) und (f) die genannten Teile identifiziert und auf Pixel-für-Pixelbasis modifiziert werden.
  13. Verfahren nach einem der vorstehenden Ansprüche, wobei das aktuelle Frame und die erhaltene Segmentierung in den Schritten (a) bis (f) verwendet werden als Bezugsframe und Bezugsframesegmentierung zum Erhalten der konsistenten Segmentierung für ein betreffendes Subsegmentframe.
  14. Verfahren nach einem der vorstehenden Ansprüche, wobei die Bildframesequenz einen Film aufweist.
  15. Verfahren nach einem der vorstehenden Ansprüche, wobei dieses Verfahren weiterhin den nachfolgenden Verfahrensschritt umfasst: das Verarbeiten von Pixeldaten des aktuellen Bildframes entsprechend der aktuellen Framesegmentierung, erhalten in dem Schritt (f) und das Ausliefern eines Signals, das die verarbeiteten Pixeldaten transportiert.
  16. Verfahren nach Anspruch 15, das weiterhin den nachfolgenden Verfahrensschritt umfasst: das direkte oder indirekte Aufzeichnen des genannten verarbeiteten Bildframes auf oder in einem Aufzeichnungsmedium.
  17. Verfahren nach Anspruch 15 oder 16, wobei der genannte Verarbeitungsschritt das betreffende Frame mit Kompression codiert, und zwar unter Verwendung der erhaltenen Segmentierung.
  18. Verfahren nach Anspruch 17, wobei das genannte codierte Frame eine Darstellung der erhaltenen Segmentierung umfasst.
  19. Verfahren nach Anspruch 17 oder 18, wobei die erhaltene Segmentierung zum verschiedenartigen Zuordnen von Bandbreite zu verschiedenen Teilen des Bildes verwendet wird.
  20. Bildverarbeitungsanordnung zum Erzeugen einer konsistenten Segmentierung in Gebiete einer Sequenz relatierter Bildframes, so dass gleiche Gruppen von Pixeln zu gleichen Gebieten in aufeinander folgenden Frames gehören, wobei diese Anordnung Folgendes umfasst:
    a) Mittel zum Empfangen aktueller Framepixeldaten für ein betreffendes Frame, Bezugsframepixeldaten und eine Bezugsframesegmentierung;
    b) Mittel zum Berechnen von Bewegungsschätzungsinformation aus den aktuellen Framepixeldaten und den Bezugsframepixeldaten um räumliche Transformationen darzustellen, die zwischen den genannten Frames auftreten;
    c) Mittel zum Erhalten einer vorhergesagten aktuellen Framesegmentierung unter Verwendung der genannten Bewegungsschätzungsinformations- und der Bezugsframesegmentierungsdaten;
    d) Mittel zum Erhalten vorhergesagter aktueller Framepixeldaten unter Verwendung der genannten Bewegungsschätzungsinformations- und der Bezugsframepixeldaten
    e) Mittel zum Identifizieren nicht einwandfrei vorhergesagter Teile der vorhergesagten aktuellen Framesegmentierung durch einen Vergleich der empfangenen aktuellen Framepixeldaten und der vorhergesagten aktuellen Framepixeldaten; und
    f) Mittel zum Modifizieren der vorhergesagten aktuellen Framesegmentierung für die identifizierten Teile durch Zuordnung wenigstens einiger Pixel derartiger Teile zu bestehenden Gebieten.
  21. Anordnung nach Anspruch 20, wobei die Modifizierungsmittel (f) Mittel zum Konstruieren und Einfügen einer neuen Segmentierung der identifizierten nicht einwandfrei vorhergesagten Teile in die vorhergesagte aktuelle Framesegmentierung.
  22. Anordnung nach Anspruch 21, wobei die Modifizierungsmittel (f) Mittel aufweisen zum vorübergehenden Zuordnen von Teilen zu denselben gebieten wie benachbarte Teile, die nicht von den Identifizierungsmitteln identifiziert wurden, und zwar vor der Konstruktion der genannten neuen Segmentierung.
  23. Anordnung nach Anspruch 21 oder 22, die weiterhin Mittel aufweist zum Mischen von Gebieten der genannten neuen Segmentierung zum Reduzieren der Anzahl neu geschaffener Gebiete vor der Einfügung in die vorhergesagte Segmentierung.
  24. Anordnung nach Anspruch 21, 22 oder 23, die weiterhin Mittel aufweist zum Mischen von Gebieten der Segmentierung nach der Einfügung der neuen Segmentierung für die identifizierten Teile, zum Reduzieren der Anzahl Gebiete in der erhaltenen Segmentierung.
  25. Anordnung nach einem der Ansprüche 20 bis 24, wobei die Identifizierungsmittel (e) weiterhin Mittel aufweisen zum Verwenden der vorhergesagten Segmentierung zum Zuordnen weiterer Teile der vorhergesagten Segmentierung als nicht einwandfrei vorhergesagt.
  26. Anordnung nach Anspruch 25, wobei die genannten Identifizierungsmittel (e) dazu vorgesehen sind, dass weitere Teile, die als nicht einwandfrei vorhergesagt zugeordnet wurden, diejenigen teile sind, die an den Grenzen von Gebieten in der vorhergesagten Segmentierung liegen.
  27. Anordnung nach einem der Ansprüche 20 bis 26, wobei die genannten Modifizierungsmittel (f) Mittel aufweisen zum Zuordnen nicht einwandfrei vorhergesagter Teile zu bestehenden Gebieten, abhängig von den Zugeordneten Gebieten benachbarter Teile.
  28. Anordnung nach Anspruch 17, wobei die Modifizierungsmittel (f) Mittel aufweisen zum Zuordnen wenigstens einige Teile der nicht einwandfrei vorhergesagten Teile zu denselben Gebieten wie einwandfrei vorhergesagte benachbarte Gebiete in Abhängigkeit von Pixeldatengradienten in dem aktuellen Frame.
  29. Anordnung nach Anspruch 28, wobei die Modifizierungsmittel (f) derart vorgesehen sind, dass nicht einwandfrei vorhergesagte Teile mit Gradienten über einer Schwelle denselben Gebieten zugeordnet werden, wie Nachbarteile in Abhängigkeit von den Zuordnungen und Gradienten der genannten Nachbarteile.
  30. Anordnung nach Anspruch 27, 28 oder 29, wobei die Modifizierungsmittel (f) dazu vorgesehen sind, dass wenigstens einige Teile der nicht einwandfrei vorhergesagten Teile denselben Gebieten zugeordnet werden wie einwandfrei vorhergesagte Nachbarteile in Abhängigkeit von Statistiken der Pixeldaten für diese Gebiete.
  31. Anordnung nach einem der Ansprüche 20 bis30, wobei die Identifizierungsmittel (e) und die Modifizierungsmittel (f) derart vorgesehen sind, dass Teile auf Pixel-für-Pixelbasis identifiziert und modifiziert werden.
  32. Anordnung nach einem der Ansprüche 20 bis 31, derart vorgesehen, dass das aktuelle betreffende Frame und die erhaltene Segmentierung durch Mittel (a) bis (f) als Bezugsframe und Bezugsframesegmentierung benutzt werden zum Erhalten der konsistenten Segmentierung für ein nachfolgendes betreffendes Frame.
  33. Anordnung nach einem der Ansprüche 20 bis 32, derart vorgesehen, dass die Bildframesequenz einen Film enthält.
  34. Anordnung nach einem der Ansprüche 20 bis 33, weiterhin mit Mitteln zum verarbeiten von Pixeldaten des aktuellen Bildframes entsprechend der aktuellen Framesegmentierung, erhalten durch die Modifizierungsmittel (f) und Mitteln zum Ausliefern eines Signals, das die verarbeiteten Pixeldaten trägt.
  35. Anordnung nach Anspruch 34, weiterhin mit Mitteln zum direkten und indirekten Aufzeichnen der genannten verarbeiteten Pixeldaten auf oder in einem Aufzeichnungsmedium.
  36. Anordnung nach Anspruch 34 oder 35, wobei das genannte Verarbeitungsmittel ein Codierer ist zum Codieren des betreffenden Frames mit Kompression, unter Verwendung der erhaltenen Segmentierung.
  37. Anordnung nach Anspruch 36, wobei das codierte Frame eine Darstellung der erhaltenen Segmentierung aufweist.
  38. Anordnung nach Anspruch 36 oder 37, weiterhin mit Mitteln zum Verwenden der genannten erhaltenen Segmentierung zur Steuerung der Zuordnung der Bandbreite zu verschiedenen Teilen des Bildes.
  39. Vom Computer verwendbares Medium, das vom Computer auslesbare Instruktionen speichert um dafür zu sorgen, dass ein Prozess in einer Computergraphikanordnung Signale verarbeitet, die aktuelle Framepixeldaten für ein betreffendes Frame. Bezugsframepixeldaten und eine Bezugsframesegmentierung definieren, wobei die Instruktionen Instruktionen enthalten um dafür zu sorgen, dass der Prozessor Folgendes durchführt:
    a) das Berechnen von Bewegungsschätzungsinformation aus den aktuellen Framepixeldaten und den Bezugsframepixeldaten um räumliche Transformationen darzustellen, die zwischen den genannten Frames auftreten;
    b) das Verwenden der genannten Bewegungsschätzungsinformations- und der Bezugsframesegmentierungsdaten zum Erhalten einer vorhergesagten aktuellen Framesegmentierung; c) das Verwenden der genannten Bewegungsschätzungsinformations- und der Bezugsframepixeldaten zum Erhalten vorhergesagter aktueller Framepixeldaten;
    d) das Identifizieren nicht einwandfrei vorhergesagter Teile der vorhergesagten aktuellen Framesegmentierung durch einen Vergleich der empfangenen aktuellen Framepixeldaten und der vorhergesagten aktuellen Framepixeldaten; und
    e) das Modifizieren der vorhergesagten aktuellen Framesegmentierung für die identifizierten Teile durch Zuordnung wenigstens einiger Pixel derartiger Teile zu bestehenden Gebieten.
EP97944071A 1996-11-13 1997-10-30 Bildsegmentierung Expired - Lifetime EP0873654B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
GB9623573 1996-11-13
GBGB9623573.4A GB9623573D0 (en) 1996-11-13 1996-11-13 Image segmentation
PCT/IB1997/001356 WO1998021893A1 (en) 1996-11-13 1997-10-30 Image segmentation

Publications (2)

Publication Number Publication Date
EP0873654A1 EP0873654A1 (de) 1998-10-28
EP0873654B1 true EP0873654B1 (de) 2006-12-27

Family

ID=10802845

Family Applications (1)

Application Number Title Priority Date Filing Date
EP97944071A Expired - Lifetime EP0873654B1 (de) 1996-11-13 1997-10-30 Bildsegmentierung

Country Status (7)

Country Link
US (1) US6173077B1 (de)
EP (1) EP0873654B1 (de)
JP (1) JP2000503509A (de)
KR (1) KR100583902B1 (de)
DE (1) DE69737141T2 (de)
GB (1) GB9623573D0 (de)
WO (1) WO1998021893A1 (de)

Families Citing this family (33)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6463168B1 (en) * 1998-12-04 2002-10-08 General Electric Company Apparatus and method for rapid connectivity processing of images
DE60028884T2 (de) * 1999-03-18 2007-01-25 Koninklijke Philips Electronics N.V. Videosignalverarbeitung
US6553069B1 (en) * 1999-06-17 2003-04-22 Samsung Electronics Co., Ltd. Digital image segmenting method and device
JP4224748B2 (ja) * 1999-09-13 2009-02-18 ソニー株式会社 画像符号化装置および画像符号化方法、画像復号装置および画像復号方法、記録媒体、並びに画像処理装置
US7027665B1 (en) * 2000-09-29 2006-04-11 Microsoft Corporation Method and apparatus for reducing image acquisition time in a digital imaging device
WO2002051157A2 (en) * 2000-12-19 2002-06-27 Pulsent Corporation Adaptive transforms
US6901169B2 (en) * 2001-02-01 2005-05-31 At & T Corp. Method and system for classifying image elements
US6965379B2 (en) 2001-05-08 2005-11-15 Koninklijke Philips Electronics N.V. N-view synthesis from monocular video of certain broadcast and stored mass media content
US7120277B2 (en) * 2001-05-17 2006-10-10 Koninklijke Philips Electronics N.V. Segmentation unit for and method of determining a second segment and image processing apparatus
US20020191699A1 (en) * 2001-06-12 2002-12-19 O'brien Royal Detection system and method for enhancing digital video
US7248741B2 (en) * 2002-01-09 2007-07-24 Hiroshi Akimoto Video sequences correlation and static analysis and scene changing forecasting in motion estimation
US6925125B2 (en) * 2002-01-09 2005-08-02 Hiroshi Akimoto Enhanced aperture problem solving method using displaced center quadtree adaptive partitioning
JP2005535028A (ja) * 2002-07-31 2005-11-17 コーニンクレッカ　フィリップス　エレクトロニクス　エヌ　ヴィ セグメント化のためのシステムおよびセグメント化方法
GB0221144D0 (en) * 2002-09-12 2002-10-23 Snell & Wilcox Ltd Image processing using vectors
WO2005125218A1 (en) * 2004-06-16 2005-12-29 Koninklijke Philips Electronics N.V. Prediction error based segmentation refinement within a forward mapping motion compensation scheme
US7916173B2 (en) 2004-06-22 2011-03-29 Canon Kabushiki Kaisha Method for detecting and selecting good quality image frames from video
AU2005202715B2 (en) * 2004-06-22 2008-04-24 Canon Kabushiki Kaisha A Method for Detecting and Selecting Good Quality Image Frames from Video
WO2006082541A2 (en) * 2005-02-07 2006-08-10 Koninklijke Philips Electronics N.V. Segmentation of an image
US7787885B1 (en) 2006-08-10 2010-08-31 Nextel Communications Inc. Walsh code expansion in wireless communications systems
US7940977B2 (en) * 2006-10-25 2011-05-10 Rcadia Medical Imaging Ltd. Method and system for automatic analysis of blood vessel structures to identify calcium or soft plaque pathologies
US7983459B2 (en) 2006-10-25 2011-07-19 Rcadia Medical Imaging Ltd. Creating a blood vessel tree from imaging data
US7940970B2 (en) * 2006-10-25 2011-05-10 Rcadia Medical Imaging, Ltd Method and system for automatic quality control used in computerized analysis of CT angiography
US7873194B2 (en) * 2006-10-25 2011-01-18 Rcadia Medical Imaging Ltd. Method and system for automatic analysis of blood vessel structures and pathologies in support of a triple rule-out procedure
US7860283B2 (en) 2006-10-25 2010-12-28 Rcadia Medical Imaging Ltd. Method and system for the presentation of blood vessel structures and identified pathologies
KR101370286B1 (ko) * 2007-04-06 2014-03-06 삼성전자주식회사 레지듀얼 블록의 변형을 이용한 영상 부호화, 복호화 방법및 장치
US8320665B2 (en) * 2009-05-13 2012-11-27 Tata Consultancy Services Ltd. Document image segmentation system
EP2360927A3 (de) * 2010-02-12 2011-09-28 Samsung Electronics Co., Ltd. Bildcodier-/-decodiersystem mit graphenbasierter Pixelvorhersage und Codiersystem und -verfahren
US9665767B2 (en) 2011-02-28 2017-05-30 Aic Innovations Group, Inc. Method and apparatus for pattern tracking
US8358823B2 (en) * 2011-03-30 2013-01-22 Mitsubishi Electric Research Laboratories, Inc. Method for tracking tumors in bi-plane images
EP3080590B1 (de) * 2013-12-12 2020-08-12 G.M.S. Global Mobile Solutions Ltd. Heimtestvorrichtung
GB2576846A (en) * 2017-04-21 2020-03-04 Zenimax Media Inc Systems and methods for game-generated motion vectors
CN107609057B (zh) * 2017-08-25 2020-12-22 百度在线网络技术（北京）有限公司 一种获取商标图像的文字数据的方法与装置
DE102018100895A1 (de) * 2018-01-16 2019-07-18 Zoe Life Technologies Holding AG Währungseinheiten für Wissen

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE69032674T2 (de) 1990-01-12 1999-05-12 Koninkl Philips Electronics Nv Bereichs- und Textkodierung
GB2243512A (en) 1990-04-23 1991-10-30 Philips Electronic Associated Bandwidth reduction of picture signals; predictive encoding of edge map signals
GB9215102D0 (en) * 1992-07-16 1992-08-26 Philips Electronics Uk Ltd Tracking moving objects
JP2576771B2 (ja) * 1993-09-28 1997-01-29 日本電気株式会社 動き補償予測装置
US5592226A (en) * 1994-01-26 1997-01-07 Btg Usa Inc. Method and apparatus for video data compression using temporally adaptive motion interpolation
US5608458A (en) * 1994-10-13 1997-03-04 Lucent Technologies Inc. Method and apparatus for a region-based approach to coding a sequence of video images
JP3663645B2 (ja) * 1994-11-18 2005-06-22 ソニー株式会社 動画像処理装置とその方法
US5778192A (en) * 1995-10-26 1998-07-07 Motorola, Inc. Method and device for optimal bit allocation between different sources of information in digital video compression
US5774591A (en) * 1995-12-15 1998-06-30 Xerox Corporation Apparatus and method for recognizing facial expressions and facial gestures in a sequence of images

Also Published As

Publication number Publication date
JP2000503509A (ja) 2000-03-21
EP0873654A1 (de) 1998-10-28
GB9623573D0 (en) 1997-01-08
DE69737141T2 (de) 2007-10-04
DE69737141D1 (de) 2007-02-08
KR100583902B1 (ko) 2006-11-30
US6173077B1 (en) 2001-01-09
KR19990077203A (ko) 1999-10-25
WO1998021893A1 (en) 1998-05-22

Similar Documents

Publication Publication Date Title
EP0873654B1 (de) Bildsegmentierung
US5572258A (en) Motion compensation estimating device and method achieving improved motion compensation
KR100918544B1 (ko) 화상 처리 장치 및 방법, 및 촬상 장치
Choi et al. Spatio-temporal video segmentation using a joint similarity measure
US6335985B1 (en) Object extraction apparatus
US4924310A (en) Method for the determination of motion vector fields from digital image sequences
Wang et al. Representing moving images with layers
EP0737012B1 (de) Verfahren zur Segmentierung und Schätzung des Bewegungsfeldes eines sich bewegenden Objektes
US7760911B2 (en) Method and system for segment-based optical flow estimation
KR100609249B1 (ko) 영상 오브젝트 트랙킹 방법과 대응 시스
US6687405B1 (en) Image segmentation
US5854856A (en) Content based video compression system
Bao et al. High-order model and dynamic filtering for frame rate up-conversion
US20200296401A1 (en) Method and Apparatus of Patch Segmentation for Video-based Point Cloud Coding
EP3718306B1 (de) Clusterverfeinerung für textursynthese in der videocodierung
EP1833243B1 (de) Bildverarbeitungsvorrichtung und -verfahren und Bildaufnahme-Vorrichtung
Tzovaras et al. 3D object articulation and motion estimation in model-based stereoscopic videoconference image sequence analysis and coding
Malassiotis et al. Object-based coding of stereo image sequences using three-dimensional models
US6262409B1 (en) Method for the detection of the relative depth of objects in an image from a pair of images
US8582882B2 (en) Unit for and method of segmentation using average homogeneity
Wang et al. Layered representation for image sequence coding
Doulamis et al. Efficient unsupervised content-based segmentation in stereoscopic video sequences
Gu et al. Morphological moving object segmentation and tracking for content-based video coding
Pardas et al. Motion and region overlapping estimation for segmentation-based video coding
Schreer et al. Hybrid recursive matching and segmentation-based postprocessing in real-time immersive video conferencing

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB

17P Request for examination filed

Effective date: 19981123

17Q First examination report despatched

Effective date: 20020918

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 69737141

Country of ref document: DE

Date of ref document: 20070208

Kind code of ref document: P

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070928

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20080328

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20081028

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20081211

Year of fee payment: 12

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20090501

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20100630

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091102

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20091030