GB2350515A - Video encoder - Google Patents

Video encoder Download PDF

Info

Publication number
GB2350515A
GB2350515A GB0012273A GB0012273A GB2350515A GB 2350515 A GB2350515 A GB 2350515A GB 0012273 A GB0012273 A GB 0012273A GB 0012273 A GB0012273 A GB 0012273A GB 2350515 A GB2350515 A GB 2350515A
Authority
GB
United Kingdom
Prior art keywords
image
encoding
encoded
areas
accordance
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB0012273A
Other versions
GB0012273D0 (en
Inventor
Kevin Michael Mckoen
Anthony Richard May
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Motorola Solutions UK Ltd
Original Assignee
Motorola Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Motorola Ltd filed Critical Motorola Ltd
Publication of GB0012273D0 publication Critical patent/GB0012273D0/en
Publication of GB2350515A publication Critical patent/GB2350515A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/507Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction using conditional replenishment
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • H04N19/14Coding unit complexity, e.g. amount of activity or edge presence estimation
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146Data rate or code amount at the encoder output
    • H04N19/149Data rate or code amount at the encoder output by estimating the code amount by means of a model, e.g. mathematical model or statistical model
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H04N19/159Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/20Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
    • H04N19/23Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/37Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability with arrangements for assigning different transmission priorities to video input data or to video coded data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

A method for image encoding is provided having the steps of providing an image containing image types; deriving a partition of the image; subdividing the partition into data blocks; classifying the subdivisions in accordance with the image types present in the corresponding parts of the image, and encoding the image blocks in accordance with the derived classification. An alternative embodiment of said method has the steps of: determining the areas of the image that are to be transmitted less frequently than other areas and randomly transmitting encoded data representing those areas over time such that during a given time period substantially all of the determined areas are transmitted at least once. Another embodiment of said method has the steps of: classifying the image types according to the available coding options and encoding respective parts of the image corresponding to respective identified types in accordance with the respective coding option. These methods permit the enhanced viewing of an image sequence, particularly after transmission over a transmission channel of limited bandwidth, since no separate carrier signal is necessary to prioritise the selected data blocks. The invention may be incorporated in a surveillance system.

Description

2350515 An image encoding method and apparatu
Field of the invention
This invention relates to an image encoding method and apparatus for encoding image information particularly, but not exclusively, for subsequent transmission from a first location to a second remote location via a communication link. The invention particularly, but not exclusively, is applicable to video image transmission.
Background to the invention is Video camera surveillance systems are becoming a familiar sight on the streets of our towns and other areas requiring protection from the criminal fraternity. The more sophisticated of these systems may employ radio communication links to a central control location where a human operator may view the monitored scene displayed on a visual display unit. Radio communication is preferred because of its convenience, although hard wire and optical fiber links are also used.
Radio communication, whilst convenient, has associated with it problems of bandwidth (indeed some hard-wire links also have this problem). In communications terms, the link is likened to a "pipe" and like pipes for carrying water, the "pipe" will have a certain transmission capacity typically expressed in bits of information transmissible per second. For radio communication, the "pipe" is relatively narrow, that is to say, the amount of information derived from a scene being monitored would take a long time to transmit. To alleviate this problem, video "compression techniques" are used with examples of such techniques being MPEG4 and ITU H.263.
ITU H.263 is a video compression scheme in which an encoder partitions the image or video into 16 x 16 pixel macroblocks further comprised of four 8 x 8 pixel blocks. These blocks are then encoded but the encoder treats all parts of the image equally even if the viewer of the sequence is only interested in certain parts of the image.
MIPEG 4 is a video compression technique in which video objects are individually encoded. However, this type of technique is proposed for use in systems offering relatively high data rates because the information within each object has to transmitted together with its definition.
A prior art system is shown in figure I and it includes a video camera I connected to a segmentation module 2. The segmentation module 2 processes video information produced by the camera I to generate a rate control signal which is input to an MPEG4 codec 3. The codec 3 encodes video information received from the camera 1 in response to a control signal and passes it to a radio modem 4. The radio modem 4 transmits 2 encoded data via antenna 5 and a radio link to a receiving antenna 6. The antenna 6 is connected to a radio modem 7 which passes the encoded data to decoder 8. The decoded data is displayed on a visual display unit (VDU) 9 where it may be viewed by the user 10.
The segmentation module 2 splits the scene being monitored into two main parts. A first part being the object of most interest (typically this will be a moving object), in this case a person present in the scene and a second part of lower interest to the viewer 10 namely the background of the scene. (In some systems there may be more than two main parts (as will be readily appreciated by the skilled exponent of the art). The video information concerning each part is sent as segmentation information to the NTEG codec 3 for onward transmission. This incurs a bandwidth overhead, which may not be acceptable, in that it is necessary to send both the video, and the segmentation information.
Brief summary of the invention
According to the invention there is provided an image encoding method comprising the steps of. providing an image containing image types; deriving a partition representation of the image; subdividing the partition representation into subdivisions; classifying the subdivisions in accordance with the image types present in corresponding parts of the image; and encoding the image in accordance with the derived corresponding classification.
3 By subdividing the partition representation of the image into subdivisions and classifying the subdivisions according to type of images present, and then encoding the image accordingly, the most appropriate encoding rate is used for parts of the image and indeed the rate may be, for some parts, zero. That is to say the image is not encoded or an abbreviated code sent to indicate that previously sent information is to be re-used. This reduces the amount of information to be later processed or transmitted. Whilst the described embodiments are concerned with systems in which transmission of the image is achieved by radio waves, the invention is applicable to other transmission systems or indeed where processes, other than transmission, are later carried out on the encoded information. Thus, the invention is not limited to methods which require transmission of the encoded information.
lt is envisaged that the partition representation of the image may take many forms but the preferred embodiment utilises a segmentor to produce a segmented image. In the specific embodiments, the subdivisions are blocks within a representation of the image in the form of a particular memory structure referred to as an array. The specific embodiments refer to this set of subdivisions as an object macroblock map.
According to a farther aspect of the invention, there is provided a method of encoding image information comprising providing an image containing image types; deriving a partition representation of the image; subdividing the partition representation into subdivisions; classifying the subdivisions in accordance with available coding options 4 and the image types present; and encoding the image in accordance with the derived corresponding classification. In this way, the most appropriate level of classification can be made in the sense that the number of types utilised is related to the available coding options. By coding options it is meant the encoding rate or quantization or other variations available in encoding techniques to be used. In the specific embodiments, some types of image are objects of particular interest, typically these will be moving, and other types of image include background which changes relatively slowly. These are encoded and then transmitted at different rates. In the case of background, it is necessary to transmit this information relatively infrequently in comparison with the transmission of information concerning the moving object. The object could be a person or other object such as a vehicle.
According to another aspect of the invention there is provided a method of encoding image information comprising the steps of determining areas of the image that are to be encoded less frequently than other areas and randomly encoding data representing those areas over time such that during a given time period substantially all of the determined areas are encoded.
The advantage of this aspect of the invention is that the areas that are to be encoded less frequently are encoded in a random manner. Where, for example, the areas are background this would mean that the background is refreshed over a number of frames when the encoded information is transmitted and then displayed.
Preferably, the method includes comprising the step of providing a list of randomly ordered image area identifiers and wherein a decision on whether to encode a particular image area is made by reference to the list. In this manner, an image area can be encoded at different rates over time or indeed the same rate for a number of times. For example, the image type may be freshly uncovered background which is uncovered as an object moves in the field of view. It will be advantageous to initially encode this at a high rate and then over time encode this at a different rate or less frequently. Background is, of course, parts of the image that are less important to the user than others. Thus, these areas need not be confined to the 'back" of the image.
Preferably, the method includes the steps of providing an image containing image types, deriving a partition representation of the image, subdividing the partition representation into subdivisions, and wherein the decision to encode at least one first image area is made by comparing the identity of a subdivision with the contents of the list.
Whilst it would be possible for the area to be encoded to not correspond to the subdivision, preferably, in the decision to encode step, the identity of the subdivision corresponds to the at least first image area. Further, it is envisaged that more than one area may be encoded in response to the decision and that at least one additional area is encoded on the basis of the decision to encode step. Preferably, the at least one additional area is in fixed spatial relationship to the at least one first image area. This spatial relationship may be such that areas of the image not immediately adjacent may be encoded on the basis of one decision. Preferably, however the least one additional area is 6 an immediate neighbouring area of the at least one first image area. In this way, blocks of the image may be treated at the same time.
In some arrangements it may be preferable to vary the classification of at least one of the image types present in the image. In this way the encoding of that type may vary over time. For example, in the case of background in an image containing a moving object some areas of background may be 'Treshly" uncovered as the object moves. Thus, the background may be classified initially as such and then encoded at a first, for example, high rate and then the classification changed over time and different, for example lower, 10 coding rates used.
This invention also provides apparatus.
Brief description of the drawings
A specific embodiment of the invention will now be described with reference to the drawings in which:
Figure 1 is a prior art figure;
Figure 2 is a schematic block diagram of a video transmission apparatus in accordance with the invention; Figure 3 is a block diagram of a segmentor module present in the apparatus of Figure 2; Figure 4 is an explanatory diagram illustrating the segmentor module's operation; Figure 5 shows in greater detail an OMB module present in the apparatus of Figure 2; Figures 6 and 7 are explanatory diagrams illustrating the operation of the OMB module of Figure 5; and Figure 8 is an explanatory diagram of how the codec rate is varied utilising the OMB module.
Detailed descriptio
A surveillance system operating in accordance with the invention comprises a remote part and a local part as is shown in figure 2.
The remote part comprises a video camera 31 monitoring a scene 30, a segmentor module 32, an Object Macroblock module 33(hereinafter abbreviated to OMB), a codec 34, a radio modem 35 and antenna 36.
The video camera 31 is both of conventional construction and operation and, thus, will not be described in greater detail. An output of the camera 31 is connected to the segmentor module 32 and also to the codec 34. An output of the segmentor module 32 is operably coupled to the OMB 33. The OMIB 33 provides a control output coupled to the codec 34. The codec 34 provides an output for encoded video data, which is coupled to the radio modem 35.
Before elaborating further on the components of the remote part, the components forming the local part will be introduced but not described in detail because they are of 10 conventional construction and operation.
The local part comprises a control console including a visual display unit 40 (VDU), a decoder 39 and a radio modem 38 with associated antenna 37. The radio modem 38 is coupled to the decoder 39 and that, in turn, is coupled to the console 40.
The local and remote part communicate via a radio link established via the radio modems 35 and 38 in a manner well known to a man skilled in the art which will not be described further.
The operation of the segmentor module 32 of the remote part is conventional in manner but will be briefly described for the purposes of understanding the inventive aspects of the system. Figure 3 shows the segmentor module 32 in greater detail. It comprises a first processor 42 in the form of a microprocessor, a memory 43 and a rules database 44, 9 and a pre-processor 47. The segmentor module 32 also has an input 45 and an output 46. The pre-processor 47 is coupled to the memory 43 by the usual databus arrangement. The pre-processor 47 processes the data received from the camera 31 via input 45 into a form suitable for storage in the memory 43. The first processor 42 is coupled to the memory 43 and the rules database 44 by a conventional databus arrangement. Whilst, for convenience, the rules database is shown as a separate block, in practice it could be a block allocated within the memory 43. The processor 42 is also coupled to the output 46.
The segmentor module 32 is provided with video data comprising video scans from the video camera 31 via its input 45. The scans are shown in figure 4 labeled as 50. In this case the scans are encoded in accordance with ITU recommendation 601 although alternative formats could be used. Each scan comprises data concerning individual pixels and the data includes chrominance and luminance information. Each data block is represented by a square. Each of the scans passes to the pre-processor 47 from where it is input into an image store in the memory 43. Once the complete image has been stored, the processor 42 processes the stored image into a segmented form 70a, 70b or, as is referred to in the art, into the segmentation map or partition image and thus provides a partition representation.
The segmentation process relies on the application by the processor 42 of the rules stored in the rules database 44. In essence, the rules determine how to divide the image into regions or partitions. For example, the rules may determine that a particular region is moving, that is to say, a moving object is present. The segmentor produces what are called in the art segmentation maps 70 and these are partition representations. These are depicted in the explanatory figure as black and white images where the white region indicates that the corresponding part of the image is moving and the black region indicates that the corresponding part is stationary (or nearly so) background. The segmentation maps 70 are output from the segmentor module 32 to the OMB 33.
The 0MB 30 is shown in greater detail in Figure 5a and comprises a processor 71, a rules database 72, a memory 73, an input 75 and an output 76. The input 75 couples the segmentor module 32 to the processor 7 1. The processor 71 is also coupled to the rules database 72 and the memory 73 by a databus. The control signal is coupled via the output 76 to the codec 34. The processor 71 is a conventional microprocessor suitably programmed to perform the OMB functions and the rules database and memory73 are provided by random access memory semiconductor chips.
is The processor 71 can therefore be represented in functional terms as shown in Figure 5b as comprising three major functional blocks; an OMB generator, an OMB refiner and an OMB rate controller.
Returning to Figure 5a, the codec 34 is conventional in construction and comprises a rate controller 77, a compressor 78, an input 79 and output 80, and a control signal input 76 coupled to the output of processor 71 of OMB module 72. The input is coupled to the video camera and receives the video data. The input 79 is coupled to the compressor 78.
The compressor 78 is coupled via the output 80 to the radio modem 35. The control signal is coupled to the compressor. The rate controller 77 also provides a control signal to the compressor 78 and has a feedback path therefrom.
The OMB 30 operates in the following manner as illustrated by Figure 6.
In a first stage shown in Figure 6a, utilising the OMB generator, the large number of pixels of the segmentation maps 70 are grouped together to form blocks of 16 by 16 pixels. This renders the image to a macroblock image having for example an eleven by nine arrangement of macroblocks for Quarter resolution Common Intermediate Format (QCIF). The macroblock image is then stored in the memory 73. In the next stage shown in Figure 6b, the processor 71 processes the macroblock image by application of the rules to each of the macroblocks in turn to classify the block as to type and priority. The type and priority information is held in the macroblock concerned. Thus, the map produced is a memory structure.
The way in which type is allocated to each macroblock will now be illustrated with reference to figure 7. In the figure two types are depicted; Type 0 shown in dotted in-fill and Type I having no in-fill. Type 0 equates to background in the image (that is to say a part of relatively little interest to the user) and typel equates to an object i. e. a part of relatively high interest to the user. In this case the object is a person in the area being viewed. Thus, as is shown, in the upper part of the figure, where the person is shown superimposed on the macroblock structure, the blocks will record type according to the presence or not of an object or background. The type present is determined according to
12 the rules, for example, parts of the image that move with a velocity above a certain threshold or having a greater than a certain threshold of Infra Red intensity will be determined as objects.
The process also applies a priority value to each macroblock. This allows a further classification to be carried out within each type. For example, there may be more than one moving object in the field of view. They will be recorded as type 1 and then rated with different priorities on a basis of speed of motion or some other criteria as defined in the rules. A higher priority may be allocated to the faster moving object.
The way in which the stored OMB map is processed will now be explained. Having derived and stored the OMB map in the memory 73, the processor 71 can access it and derive from it the values of type and priority on a block by block basis. It will then generate on the basis of the values, the type, the priority or a combination of both and is hence, the appropriate control signals to pass to the compressor in the codec 34, as depicted in figure 8. The compressor then varies its encoding rate accordingly. For example, the control signals could instruct the compressor to skip or not skip encoding of the image area corresponding to the macroblock or to use a specific quantizer value or skip or not skip the whole frame. The rate varies such that the more important parts of the 20 image area encoded at greater resolution, for example, those parts corresponding to an object rather than background.
13 Thus, taking the first row R1 of the OMB map depicted in figure 8, the first and second macroblocks 90, 91 are type 0 and low priority, that is, they are background. The rate of the codec is then set to level 1 which is a relatively low level by the processor 71 placing an appropriate control signal on the input of codec 34. The next two macroblocks 92 and 5 93 are type 1 having priority 1 (there being a part of a person present in that region). Thus the codec rate is set to level 2, an intermediately high level. The next macroblock 94 is background and therefore is of type 0 and priority 0. The codec rate is then set to level 1. In the final macroblock of the row RI, an object is present of a fast moving type, for example a vehicle (represented in the figure by a star). The macroblock therefore specifies type 1 (object) and a priority 2. The codec is then set to the highest rate level 3.
It will be appreciated that the process is clocked such that the control signal applied to the codec 34 is synchronised with the corresponding part of the video scan passed to the codec 34.
The encoded information is then passed to the radio modem 35 and transmitted to the receiving radio modem 38. The encoded information is then decoded by decoder 39 and then displayed at the VDU of the console 40.
There are various other ways in which the system may utilise the OMB map to compress the data to be transmitted and these techniques may be used singly or in combination.
14 Option 1 For a first few frame transmissions all video information is sent whatever the type of image data. Thereafter, background information i.e. type 0 is only transmitted every so many frames, for example, every seven frames. This is possible because background information is only slowly changing and does not require updating on a frequent basis and, of course, the background information is only of low interest to the user. This results in a reduction in the data to be transmitted because the codec is instructed by the control signal from the OMEB rate controller (Figure 5b) to transmit blocks of dummy coded data at a low rate. (The dummy data having less information than a true data block).
Option 2 The rate is controlled according to the priority allocated in the relevant macroblock. Thus, two objects of the same type may be encoded at different rates depending on the priority value stored in the macroblock.
Option 3 An object of the same type but different priority may be transmitted at different rates in terms of refreshes per number of frames. For example, an object of low priority might be relatively slow moving and will thus require fewer image updates per given set of frames in comparison to a fast moving object which might be allocated a higher priority.
In alternative embodiments of the invention, the type that is indicated in the macroblock map may be used to select an appropriate image treatment function. In a preferred alternative embodiment utilising the earlier described apparatus the following image treatment functions are used.
Background function (BF) for type T=0-7.
Object Function (OF) for type T=16 upwards. Uncovered Background Function (UBF) for type T= 8 to 15.
Background function (BF)
In a first step a randomiser list memory structure is set up and stored in the memory by the processor 71 of the OMB 30 module.
In a second step, the processor populates the randomiser list with macroblock identifiers in a random repeating sequence.
In the next step, the OMB map is accessed macroblock by macroblock as earlier described. If the type of the macroblock is T=O then the accessed macroblock identifier is compared with those in the randomiser list. If there is a match then the video data 16 corresponding to the macroblock is transmitted at the appropriate rate for a macroblock of type background. If there is not a match then that macroblock does not result in a transmission. The identifier is then deleted from the list. When the list is empty, a new random repeating sequence is generated and used to populate the randomiser list.
The significance of this use of a randomising list is that the background (T = 0) is sent on a random basis. Thus, the background is refreshed over a period of frames. That is to say, all parts of the background are eventually transmitted.
Uncovered Background Function (UBF)
As objects move within the field of view it will appreciated that parts of the background hidden by the objects will be uncovered. It is important that such fresh image information is transmitted sooner than it might otherwise be using the aforementioned is randomiser list. This is achieved by entering the type in the macroblock as type T=15 when the background is first detected as being uncovered. Initially the OMB map is created by the 01M generator according to the segmentation information. For the purpose of this example, it has type values of.
0 background types (range 0-7) 8 uncovered background types (range 8-15) 16 object types (range 16-255, for 8 bit map)
17 The processor then processes the present OMB map according to rules and information from previous OMB map.
In short:
1. If initially generated OMB map contains 'uncovered background' (type=8), these are set to type 15 in current, processed, OMB map. 2. For all initial 'background' (type 0), the previous OMB map retained in memory is checked. If type was greater than or equal to 8, decrement the type from previous OMB map and place it in current, processed, OMB map.
To explain this process further, let us define the previous OMB map as Map.last having macroblocks containing information on tile type as T.last. The new 0MB map Map.current is to be populated by new values for the types detected in each area corresponding to the particular macroblock. This process involves determining a detected type, say, T.detect and refining this by reference to the previous OMB map, that is to say, Map.last, and in particular to the type T.last of the corresponding macroblock.
Thus the following process is carried out.
For each macroblock of OMB Map. current do:
Determine T.detect in the corresponding part of the image area; 18 If T.detect = 16 then enter T=16 in macroblock If T.detect is uncovered background then enter T= 15 in macroblock If T.detect <8 then check T. last if T.last > 7 then enter T= T.last -1 in macroblock
The uncovered background is encoded by the processor using its priority value to command the compressor which 'rate' to use.
In this way, newly uncovered background is always set to 15. Unless it becomes uncovered background again, it will naturally decrement or decay into background after 8 repeat encodes. This provides a progressive refinement.
Thus it will be appreciated that uncovered background will be transmitted at the appropriate rate indicated in the macroblock for a number a frames and then it will be treated as normal background and transmitted every seventh or so frames.
A further advantageous embodiment of the invention will now be described. The apparatus is as before, as is, its method of operation. However, in this embodiment motion estimation is carried out by the codec.
In an effort to further reduce the data for transmission, conventional codecs utilise a process called motion estimation. Simplified, this process involves comparing blocks between subsequent frames to determine the motion of parts of the image. Having 19 determined the motion it is possible to infer that the vector of the motion links two image blocks that are one and the same object in reality. Hence if image information on that block has already been transmitted the block need only be displaced by the motion vector (this being known as motion compensation). This motion estimation is carried out for the 5 whole image.
In accordance with this embodiment the macroblock map is used to disable this motion estimation operation by the codec when the macroblock map indicates that the type is of normal background. This saves significant amounts of processing time by the codec which no longer has to work out the motion vectors of background. Thus, the processor 71 will upon detection of type T=O send a control signal to the codec 34 to cancel any motion correction process to be carried out on that part of the image.
It will be appreciated that the described systems provide different transmission rates for the image information to be sent. The embodiments could be enhanced by firstly determining the rates available and then classifying the image according to a corresponding number of types. This could be done in an adaptive manner, as say the available rates were to change the types available for classification could also change.
In the described embodiments it is clear that the background information has a low priority. However, this could change in an adaptive manner. For example, where there are no moving objects in the image it would be possible to utilise bandwidth allocated in anticipation of there being such objects for the transmission of background. At any given data rate and quality there is a limit to the macroblocks that can be encoded independent of the information within the blocks. The number required for sending the object or objects may be less than the maximum available andtherefore these may be used for sending of background. This decision will be carried out by the rate controller 77.
In alternative embodiments, the randomiser list could be used such that instead of one corresponding image area only being encoded adjacent or neighbouring image areas could be encoded as well.
21

Claims (25)

Claims
1. An image encoding method comprising the steps of. providing an image containing image types; deriving a partition representation of the image; subdividing the partition representation into subdivisions; classifying the subdivisions in accordance with the image types present in corresponding parts of the image; and encoding the image in accordance with the derived corresponding classification.
2. A method as claimed in claim 1 wherein the partition representation of the image is a segmented representation.
3. A method as claimed in claim 1 wherein the step of encoding the image for at least one classification comprises randomly encoding the respective parts of image over more than one frame.
4. A method as claimed in claim 3 wherein substantially all of the parts of the image having the at least one classification are encoded over a predetermined number of frames.
20.
5. A method as claimed in claim 3 or claim 4 wherein a randomiser list is generated randomly populated with identifiers of the subdivisions of the at least one 22 classification and the decision to encode a respective image part is made with reference to the randomiser list.
6. A method as claimed in any preceding claim wherein respective coding options are allocated to the respective classifications from a set of available coding options and the respective parts of the image corresponding to the respective identified types are encoded in accordance with the respective coding options.
7. An image encoding method comprising the steps of: providing an image containing image types deriving a partition representation of the image; subdividing the partition into subdivisions; classifying the subdivisions in accordance with available coding options and the image types present; and encoding the image in accordance with the derived corresponding classification.
is
8. A method of encoding image information comprising the steps of determining areas of the image that are to be encoded less frequently than other areas and randomly encoding data representing those areas over time such that during a given time period substantially all of the determined areas are encoded.
9. A method as claimed in claim 8 finther comprising the step of providing a list of randomly ordered image area identifiers and wherein a decision on whether to encode a particular image area is made by reference to the list.
23
10. A method as claimed in claim 9 comprising the steps of:
providing an image containing image types; deriving a partition representation of the image; subdividing the partition representation into subdivisions; and wherein the decision to encode at least one first image area is made by comparing the identity of a subdivision with the contents of the list.
11. A method as claimed in claim 10 wherein in the decision to encode step the identity of the subdivision corresponds to the at least first image area.
12. A method as claimed in claim 11 wherein at least one additional area is encoded on the basis of the decision to encode step.
13. A method as claimed in claim 12 wherein the at least one additional area is in fixed spatial relationship to the at least one first image area.
14. A method as claimed in claim 13 wherein the least one additional area is an immediate neighbouring area of the at least one first image area.
15. A method as claimed in any one of claims 8 to 14 wherein the areas to be less frequently encoded are areas of the image that are changing less frequently than others.
24
16. A method as claimed in any one of claims 10 to 15 wherein the subdivisions are classified in accordance with the image types present in corresponding parts of the image and the image is encoded in accordance with the derived corresponding classification.
17. A method as claimed in any preceding claim wherein the classification of at least one image type is varied over time.
18. A method as claimed in claim 17 wherein the image type is background.
19. A method as claimed in claim 18 comprising the further step of determining the presence of uncovered background in an image; allocating a first classification to the uncovered background and varying the classification over time such that the uncovered background is encoded at different rates.
20. An image encoding method substantially as hereinbefore described with reference to the accompanying drawings.
2 1. Apparatus for encoding an image comprising: means for providing an image containing image types; means for deriving a partition representation of the image; means for subdividing the partition representation into subdivisions; means for classifying the subdivisions in accordance with the image types present in corresponding parts of the image; and means for encoding the image in accordance with the derived corresponding classification.
22. Apparatus for encoding an image comprising: means for determining areas of the image that are to be encoded less frequently than other areas and means for randomly encoding data representing those areas over time such that during a given time period substantially all of the determined areas are encoded.
23. Apparatus for encoding an image substantially as hereinbefore described with reference to and as illustrated by the drawings.
24. Apparatus for transmitting image information utilising a method as claimed in any preceding claim.
25. Apparatus for transmitting image information including apparatus for encoding as claimed in any preceding claim.
26
GB0012273A 1999-05-24 2000-05-19 Video encoder Withdrawn GB2350515A (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB9912079A GB9912079D0 (en) 1999-05-24 1999-05-24 An image encoding method and apparatus

Publications (2)

Publication Number Publication Date
GB0012273D0 GB0012273D0 (en) 2000-07-12
GB2350515A true GB2350515A (en) 2000-11-29

Family

ID=10854074

Family Applications (2)

Application Number Title Priority Date Filing Date
GB9912079A Ceased GB9912079D0 (en) 1999-05-24 1999-05-24 An image encoding method and apparatus
GB0012273A Withdrawn GB2350515A (en) 1999-05-24 2000-05-19 Video encoder

Family Applications Before (1)

Application Number Title Priority Date Filing Date
GB9912079A Ceased GB9912079D0 (en) 1999-05-24 1999-05-24 An image encoding method and apparatus

Country Status (1)

Country Link
GB (2) GB9912079D0 (en)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1248470A2 (en) * 2001-03-29 2002-10-09 Matsushita Electric Industrial Co., Ltd. Image coding equipment and image coding program
GB2382940A (en) * 2001-11-27 2003-06-11 Nokia Corp Encoding objects and background blocks
EP1315380A3 (en) * 2001-11-17 2004-09-01 LG Electronics Inc. Object-based bit rate control method and system thereof
WO2005059827A2 (en) 2003-12-16 2005-06-30 Infocus Corporation System and method for processing image data
WO2006081634A2 (en) * 2005-02-04 2006-08-10 Barco N.V. Method and device for image and video transmission over low-bandwidth and high-latency transmission channels
US8325796B2 (en) 2008-09-11 2012-12-04 Google Inc. System and method for video coding using adaptive segmentation
EP2309747A3 (en) * 2005-03-01 2013-06-26 Qualcomm Incorporated Region-of-interest coding using RHO domain bit allocation
EP2046048A3 (en) * 2005-03-01 2013-10-30 Qualcomm Incorporated Region-of-interest coding with background skipping for video telephony
US9154799B2 (en) 2011-04-07 2015-10-06 Google Inc. Encoding and decoding motion via image segmentation
US9262670B2 (en) 2012-02-10 2016-02-16 Google Inc. Adaptive region of interest
US9392272B1 (en) 2014-06-02 2016-07-12 Google Inc. Video coding using adaptive source variance based partitioning
US9578324B1 (en) 2014-06-27 2017-02-21 Google Inc. Video coding using statistical-based spatially differentiated partitioning
US10848769B2 (en) 2017-10-03 2020-11-24 Axis Ab Method and system for encoding video streams

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997042766A1 (en) * 1996-05-06 1997-11-13 Philips Electronics N.V. Segmented video coding and decoding method and system
WO1998011730A1 (en) * 1996-09-12 1998-03-19 Btg International Limited Object-oriented video system
GB2322509A (en) * 1997-02-21 1998-08-26 Motorola Ltd Communication system
GB2342525A (en) * 1995-10-30 2000-04-12 Sony Uk Ltd Video quantisation based on image activity
GB2344954A (en) * 1998-09-25 2000-06-21 Nippon Telegraph & Telephone Determining a target number of bits for each region within an image

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2342525A (en) * 1995-10-30 2000-04-12 Sony Uk Ltd Video quantisation based on image activity
WO1997042766A1 (en) * 1996-05-06 1997-11-13 Philips Electronics N.V. Segmented video coding and decoding method and system
WO1998011730A1 (en) * 1996-09-12 1998-03-19 Btg International Limited Object-oriented video system
GB2322509A (en) * 1997-02-21 1998-08-26 Motorola Ltd Communication system
GB2344954A (en) * 1998-09-25 2000-06-21 Nippon Telegraph & Telephone Determining a target number of bits for each region within an image

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
IEEE Trans Image Processing, Vol 7, No 1, 1997, P Salembier et al, "Binary Partition...", pp 60-74 *
IEEE Trans Image Processing, Vol 9, No 4, 2000, P Salembier et al, "Binary Partition...", pp 561-576 *
Proc. Intl. Conf on Image Processing, Vol.2, 1996, F Marqueset al, "Partition coding..." pp. 935-938 *

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7305135B2 (en) 2001-03-29 2007-12-04 Matsushita Electric Industrial Co., Ltd. Image coding equipment and image coding program
EP1248470A3 (en) * 2001-03-29 2004-11-17 Matsushita Electric Industrial Co., Ltd. Image coding equipment and image coding program
EP1248470A2 (en) * 2001-03-29 2002-10-09 Matsushita Electric Industrial Co., Ltd. Image coding equipment and image coding program
EP1315380A3 (en) * 2001-11-17 2004-09-01 LG Electronics Inc. Object-based bit rate control method and system thereof
GB2382940A (en) * 2001-11-27 2003-06-11 Nokia Corp Encoding objects and background blocks
US7206456B2 (en) 2001-11-27 2007-04-17 Nokia Corporation Video coding and decoding
WO2005059827A2 (en) 2003-12-16 2005-06-30 Infocus Corporation System and method for processing image data
EP1695263A2 (en) * 2003-12-16 2006-08-30 Infocus Corporation System and method for processing image data
EP1695263A4 (en) * 2003-12-16 2010-04-21 Infocus Corp System and method for processing image data
WO2006081634A2 (en) * 2005-02-04 2006-08-10 Barco N.V. Method and device for image and video transmission over low-bandwidth and high-latency transmission channels
WO2006081634A3 (en) * 2005-02-04 2006-12-28 Barco Nv Method and device for image and video transmission over low-bandwidth and high-latency transmission channels
EP2309747A3 (en) * 2005-03-01 2013-06-26 Qualcomm Incorporated Region-of-interest coding using RHO domain bit allocation
EP2046048A3 (en) * 2005-03-01 2013-10-30 Qualcomm Incorporated Region-of-interest coding with background skipping for video telephony
US8693537B2 (en) 2005-03-01 2014-04-08 Qualcomm Incorporated Region-of-interest coding with background skipping for video telephony
US8768084B2 (en) 2005-03-01 2014-07-01 Qualcomm Incorporated Region-of-interest coding in video telephony using RHO domain bit allocation
US8325796B2 (en) 2008-09-11 2012-12-04 Google Inc. System and method for video coding using adaptive segmentation
US9924161B2 (en) 2008-09-11 2018-03-20 Google Llc System and method for video coding using adaptive segmentation
US9154799B2 (en) 2011-04-07 2015-10-06 Google Inc. Encoding and decoding motion via image segmentation
US9262670B2 (en) 2012-02-10 2016-02-16 Google Inc. Adaptive region of interest
US9392272B1 (en) 2014-06-02 2016-07-12 Google Inc. Video coding using adaptive source variance based partitioning
US9578324B1 (en) 2014-06-27 2017-02-21 Google Inc. Video coding using statistical-based spatially differentiated partitioning
US10848769B2 (en) 2017-10-03 2020-11-24 Axis Ab Method and system for encoding video streams

Also Published As

Publication number Publication date
GB9912079D0 (en) 1999-07-21
GB0012273D0 (en) 2000-07-12

Similar Documents

Publication Publication Date Title
US6339616B1 (en) Method and apparatus for compression and decompression of still and motion video data based on adaptive pixel-by-pixel processing and adaptive variable length coding
US5724451A (en) Image compression method and apparatus using conditional quadtree split sysyem
US5596659A (en) Preprocessing and postprocessing for vector quantization
EP0959627B1 (en) A motion video compression system with adaptive bit allocation and quantization
US5649030A (en) Vector quantization
KR100298416B1 (en) Method and apparatus for block classification and adaptive bit allocation
EP1249132B1 (en) A method for encoding images, and an image coder
US6351563B1 (en) Apparatus and method for coding/decoding scalable shape binary image using mode of lower and current layers
US5790695A (en) Image coding device for coding image signal to reduce the amount of the information in the image
EP3010232A1 (en) Image encoding device, image encoding method, image decoding device, and image decoding method
GB2350515A (en) Video encoder
KR19990071425A (en) Binary shape signal encoding apparatus and method_
CN108737825A (en) Method for coding video data, device, computer equipment and storage medium
US5933105A (en) Context-based arithmetic encoding/decoding method and apparatus
JPH1155664A (en) Binary shape signal encoding device
KR20020077093A (en) Image coding equipment and image coding program
EP1639830A1 (en) Early detection of zeros in the transform domain
JPH06225280A (en) Digital signal processing device
KR20020026189A (en) Efficient video data access using fixed ratio compression
US10827179B2 (en) Method for coding and decoding image parameters, device for coding and decoding image parameters and computer programs corresponding thereto
US4941053A (en) Predictive encoder for still pictures
EP0633700B1 (en) Method and device for the search for a prediction block in predictive video compression
JPH0244881A (en) Moving picture encoding system
JPH06169452A (en) Picture compression system having weighting on screen
KR100186268B1 (en) Edge compensation method and structure using virtual image region

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)