SE521021C2 - Method and apparatus for transmitting images - Google Patents

Abstract

An image (3) in digitized form shall be transmitted over a channel between a transmitter and a receiver. The channel has a limited bandwidth and the image has a less important background (R1) and also regions of particular importance, i.e. regions of interest (R2, Rn). The image is transformed into transform coefficients and compressed (21), and a mask corresponding to the regions (R1, R2, Rn) is defined in the transform domain (22). The transform coefficients are classified (23) and assigned to different segments (SG1, SG2, SGn) in accordance with the mask definition. These segments (24) are coded independently of one another to different degrees of accuracy, depending on the importance of corresponding regions (R1, R2, Rn) in the image (3). Coding results in sub-bit streams (25) which are linked together (26) with the image header (271, 272) to form a bit stream (27), which is sent to the receiver. The receiver decodes the image header and the segment information and reconstructs the mask in the transform domain, including shapes and positions of the regions (R1, R2, Rn). The image is then recreated with the aid of the mask to desired degrees of accuracy in respective regions. It is possible to define several regions (R2, Rn) with different degrees of image quality, and only those parts of the image that are of interest need be decoded.

Description

20 25 30 521 021 2 but is sufficiently similar to the original image for the human eye, for example.

In some applications, certain parts of the transferred image are more interesting than the rest of the image and it is therefore desirable to have a better visual quality of these parts of the image. And (ROI). such part is commonly referred to as the "region of interest" Applications in which this may be useful are databases or the transmission of 'case's are medical satellite images. For example, in some it is desirable or necessary that the region of interest be transferred without loss, while the quality of the rest of the image is of less interest. occasions when it is required that There are also the regions of interest extracted from the bitstream and decoded without the whole image having to be decoded.

The two Swedish patent applications SE 9703690-9 and SE 9800088-8 state how a mask can be calculated to delimit such a region of interest (ROI).

DISCLOSURE OF THE INVENTION The present invention addresses the above-mentioned problem of specifying and transferring regions of interest and background regions in the images of different qualities in the different regions during image transfer.

The basic idea for solving the problem is to transform the image and to define a mask in this transform that corresponds to the regions of interest and background.

The region definition and the transformation of the image are transferred to the image with the desired quality in the predetermined regions. a receiver, which can recreate Something more detailed includes the problem solving that the image is divided into the desired regions. The image is then transformed into some type of transform coefficient. A mask corresponding to the different regions in the image is defined in the transform domain and the coefficients are classified and assigned according to the mask definition to different segments. These segments thus belong to the corresponding regions in the picture.

The segments and coefficients are transmitted in compressed form to a receiver that can recreate both the regions of the image and the image itself with the desired image quality in the different regions.

The invention has the advantage that several different regions of interest can be defined.

Another advantage is that the different regions can have several different degrees of image quality. Another advantage is that only those parts of the image that are of vital interest to the user need to be decoded, while one can avoid decoding the whole image.

Another advantage is that the segments can be coded independently of each other.

The invention will now be described in more detail by means of preferred embodiments and with reference to the accompanying figures.

DESCRIPTION OF THE FIGURES Figure 1 shows a block diagram of a device according to the invention; Figure 2 shows with a flow diagram a part of a method according to the invention; Figure 3 shows with a flow diagram a further part of a method according to the invention; Figure 4 shows a diagram for classifying transform coefficients; Figure 5 shows diagrams for linking image segments in a bitstream; Figure 6 shows a view of an image with objects, and Figure 7 shows a diagram with graphical representation of the topology in Figure 6.

PREFERRED EMBODIMENTS for encoding and an image 3 of a Figure 1 schematically show a device transmitting images. A digital camera 1 has objects stored in digital form and the image is presented on a screen 4. The screen is connected to a computer 2 which has programs for dividing the image 3 into objects or regions, of which a background R1 and regions of interest R2 and Rn are displayed . An image encoder 5 in the computer 2 wavelet-transforms the image, at the same time performing an image compression, and generating a compressed bitstream PS1. An operator at the monitor 4 defines the regions of interest R2 and Rn. The image encoder has devices for creating a mask PS2 according to the regions and separating the bitstream to the corresponding regions R1, parts, segments, of R2 and Rn. by means of this attribute The definition also includes that the regions R1, R2, Rn in fornx of the different segments of the bitstream PS1 are coded with different degrees of accuracy. A transmitter 6 transmits the bitstream including the definition of the positions and shape of the regions R2 and Rn to a receiver 7, which is connected to a computer with an image decoder 8. This decodes the bitstream PS1 and recreates the mask definition PS2 and presents the image on a monitor 9.

The background R1 here has a relatively low accuracy while the regions R2 and Rn each have a higher accuracy. In order to describe the method according to the invention, the following definitions shall be made: coefficients in objects - A segment is defined as all the transform domains, which belong to a certain or the background in the image. The segments can then be further divided into sub-segments.

- A sub-segment is defined here as a number of coefficients in a part of the transform domain (for example a subband in the case of the wavelet transform) that is required for the reconstruction and belongs to a segment in the digitized image, see figure 4.

As mentioned above, the coefficients are classified and can When this classification is to be attributed to different segments. made, the segments are coded independently of each other to different degrees of accuracy, which gives a bitstream for each segment. These segments are then merged.

The method according to the invention which is carried out during the coding is to be described in connection with Figure 2. The digitized image 3 which is transmitted has the background R1 and the regions of interest R2 and Rn. The following steps are performed: 1. Perform a transformation of the image 3 according to step 21. This transformation is performed according to the example with a wavelet transform or DCT (Discrete Cosine Transform). 2. With the aid of the information on how the image 3 is to be divided into the background R1 and the objects R2 the digitized and Rn a mask is created according to step 22. In this case, for example, the technology described in the Swedish patent applications SE 9703690-9 and SE 9800088- 8. The mask is created in the transform domain and indicates which coefficients are required to reconstruct the different objects or the background. Different segments SG1, SG2 and SGn correspond to the background R1 and the objects R2 and Rn. The mask is used according to step 23 to classify the transform coefficients so that they belong to the different segments SG1, SG2, SGn. 4. Code the segments independently according to step 24.

This gives the number of bits required for each sub-segment, i.e. a set of sub-bit streams 25, one for each sub-segment. 5. Link the sub-bit streams according to step 26 together with the necessary bitstream information and image header information. This requires a bitstream description as follows.

This comprises subband 0 6. Transmit the linked bitstream 27. shape information 271, bitstream information 272, designated 273 and subband 1 designated 274.

The method enables the receiver to have immediate access to arbitrary parts in the image where required, as shown in Figure 3. This is possible because the information about where in the bitstream the different parts are is known.

In connection with Figure 3, a way for the decoder to work is described below: 1. Receive the bitstream 27 and decode the required information in the image header according to step 31. 2. Find and decode the required segment information, step 32. 3. Create a mask in the transform domain by to use, for example, the technique described in the mentioned patent applications SE 9703690-9 and SE 9800088-8, step 33. The mask describes which coefficients are required to reconstruct the desired objects or background. 4. Decode as required from the bitstream, step 34. segment data 5. Reconstruct the required segments, step 35. 6. Decode and display the image, step 36.

DESCRIPTION OF THE BIT CURRENT The components * in the bit stream 27 required when using the described technique will be described below.

Data Structure and Pointer Pointer A pointer is a set of symbols that define the position of a bit or byte in a bitstream or file. In computer science, many ways to describe pointers have been defined. Any suitable such method can be used here. A pointer can be defined implicitly by a rule for composing a bitstream. A pointer can be defined relative to an explicitly or implicitly determined position. A simple way of defining a pointer is to determine the number of bits between the requested position and a known reference point such as, for example, the first bit in the bitstream.

Topology description The topology description, TOP, is a set of symbols that determine the topological relationship between numbered objects and shapes. in which four objects O1, O2, O3, O4 and four shapes S1, S2, S3 and S4 This is illustrated in Figure 6 are shown. The topology in the image can be represented, for example, by a tree branch as shown in Figure 7. The nodes and edges of the tree branch can be coded in a data structure using well-known methods. P_TOP is a pointer to a topology description.

Shape description A shape description, Si, defines the appearance of a closed boundary line of an object. The shape number, i, is indicated by a topology description. Many different shape coding techniques can be used. Examples of such methods are chain coding and shape coding methods in MPEG-4. Shape descriptions can be decoded independently of each other once their respective position in the bitstream is known. P_Si is a pointer to a shape description.

Segment Description A segment description, Ti, is a compressed set of symbols that encode a segment as described above.

The segment contains a prescribed set of sub-segments. The object number, i, is indicated by a topology description. p_Ti is a pointer to a segment description.

Sub-segment description A sub-segment description, Bü, is an independently decodable sub-segment, j, of a segment description, TL, which describes, for example, the coefficients belonging to a given subband, j, as described above. p_Bü is a pointer to a sub-segment description.

Multiplexed segment description A plurality of segment descriptions, {Ti, Tj, Tk HJ, can be multiplexed into a common data structure MT (i, j, k).

This is usually done with the intention of performing simultaneous progressive transfer of a set of objects.

The data structure, MT, is called a multiplexed segment description. Several multiplexing methods can be used. p_MT is a pointer to a multiplexed segment description.

Segment multiplexing methods A simple to Example of multiplexing methods is shown in Figure 5. method is to interleave sub-segments 52 belonging to the component segments so that: MT (i, j, k) = {B10, Bjo, Bko, B11, B31, Bkl, B12, B32, Bkzm} Here, the order of the symbols corresponds to the order of the bitstream 51, with the symbols on the left being sent first.

Sub-segments in a multiplexed stream can be excluded if they are known by the decoder.

Bitstream storage format To provide instant access to any object in the image, the stored bitstream or file structure should include at least the following components: In the image header, if required: Topology description TOP Pointer to shape description {p_S1 , p_S2mp_SN} Segment description pointer {p_T0, p_T1, mp_TN} Optional pointer to sub-segment description: for each k: [O r N] r {p_Bk0l p_Bklr --- p_BkN 1 'In the actual bitstream if required {Form description1 S2, mSN} Segment description {T0, T1, mTN} A group of elements for segment description with index {k, l, m} can optionally be replaced with multiplexed segment description MT (k, l, m) N is the number of stored objects The background is the object with index O.

PROGRESSIVE TRANSFER WITH IMMEDIATE ACCESS TO ANY ACCOMPANYING OBJECT A server receives a request to send image data to a client. The image is stored in the server in the format described in the previous section. Some of the stored data structures forms, (topological information, segments and sub-segments) may have already been sent to the receiving terminal. This section describes a procedure for compiling a bitstream of the server processing the said request. 10 15 20 25 30 521 021 ll Example Request from user A simple request contains the following information: Send objects with the numbers k, l, m N and with the accuracy nk, nl, nm respectively, whereby the accuracy is the index of the highest sub-segment that was sent for each index.

Several primitive requests can be sent. They will be served in the order they are received or in an order otherwise prescribed.

Procedure for serving a request (details) Send topological information if required. TOP was sent in response to a first request for information regarding an image.

Submit all shape descriptions necessary to describe the boundary line of the requested objects.

Form descriptions already known to the decoder do not need to be sent. Using the topological tree branch in Figure 7, it is found that all shape descriptions on the same branch as the object at the same or lower hierarchical level do not need to be sent. The server knows the state of the decoder and will only send the shape descriptions that are not known by the decoder.

Send (multiplexed) sub-segment descriptions that describe the requested objects next to the requested accuracy.

Sub-segment descriptions already known by the decoder do not need to be transmitted. The user knows, for example, sub-segments {Bw, Bm, Bm, Bu} belonging to segments k.

Sub-segment description {BÜ, Bw, BW} must be sent if object k is requested next to accuracy 7. 10 15 20 25 521 021 12 EXAMPLES This section explains some examples of situations where the proposed method can be used.

Assume, according to Figure 5, that in the middle of the image R51 there is a region R52 which has the shape of a circle, which must have better quality than the area R53 outside the circle which is hereinafter referred to as the background. However, both background R53 and region R52 must be transmitted simultaneously. The following takes place: l. The original image is transformed with wavelet-transform. 2. A mask in the transform domain is then created. This mask describes the coefficients required in the transform domain to reconstruct the region R52 and the background R53. classify the coefficients of the transform domain into two The created mask was then used for segments, one for the region and one for the background. The two segments are made up of a number of sub-segments. The number of sub-segments in this example is the same as the number of subbands in the transform domain. The present situation is thus: 2.1 For the regional segment belonging to the region R52: {{r0, lr r0,2r --- r rO, i} r --- {rno_subbands, lr rno_subbands, 2r --- rno_subbands, j}} where i , j is the number of the coefficients in the different sub-segments. 2.2 For the background segment belonging to the background R53: {(130, lr bO, 2 r mbO, p} r --- r {bno__subbands, 1 r bno_subbands, 2 r --- bno_subbands, q}} where p, q is the number of coefficients in the two sub-segments 3. The two segments are then coded as follows: 3.1 For the regional segment: 10 15 20 25 4.

In 521 021 13 A shape description S, and a segment description Tr = {Br, 0rBr, lr --- rBr, no_subbands} and a set of sub-segment pointers {p_BLo, p_BL1, m, p_BLnq§Mmæms}.

For the background segment: A segment description Tb = {Bmo, Bm1, m, Bmnq§mmæmS} and a set of sub-segment pointers {P_Bb, of P_Bb, 1 f ---, P_Bb, no_subbands} - The two segments are then merged into a single bitstream , bitstream 51, as follows: <{p_Bm0, p_BL0, p_Bm1, p_BmnQ§mmæmm p__Br, no_subbands}> Br, no_subbands}> In this case, the sub-segments are merged as shown in the upper part of Figure 5 with the region subbitcreams 52 taken alternately with background bit streams.

Note that in case the receiver knows the order in which the different parts of the image were sent, the TOP field is not needed. The first part of the array, from to mp_B}> is in other words a definition of where the different image regions are located in the rest of the compressed bitstream.

The combined bitstream was then sent to the receiver. the decoder occurs as follows: The image header together with the topology, shape information and pointers are read.

The decoder can now create the same mask as the one above. 10 15 20 521 021 i 14 8. The decoder creates the segments with the underlying sub-segments. 9. The decoder starts by decoding the merged bitstream and fills in the transmitted transform coefficients in the corresponding sub-segments. 10. An inverse transform is used. ll. The image is sent and reconstructed.

The above is a way of using the proposed method. Other ways may be to merge the bitstreams in a different way. The region R52 can, for example, as a result of the lower part of Figure 5, be transferred first, the background R53. Another example may be that there is more than one region, as described in connection with Figure 6, whereby they are combined in a number of different ways.

In addition to the previously mentioned advantages, the proposed method also has the advantage that it is possible to send the shape information only when this is needed.

Claims (1)

1 0 15 20 25 521 021/5 PATENT REQUIREMENT Method for transmitting an image (3) between a transmitter (2,5,6) and. one. receiver (7.8), which. method comprises the steps of: dividing the image (3) into at least two image regions (R1, R2, Rn); encoding the image regions (R1, R2, Rn) into an encoded symbol stream (21), the encoding utilizing a symbolic representation and having predetermined accuracy levels in the image regions; and compressing the encoded symbol stream into a compressed bitstream (PS1, 27); characterized in that the method comprises the steps of: generating (22) a definition (PS2) of an outer boundary line (Si) for at least one of the image regions (R2, Rn); transmitting said definition (PS2) to the receiver (7): transmitting the compressed bitstream (PS1, 27) to the receiver (7,8); and decoding (33,34) in the receiver. using the said definition. Method according to claim 1, characterized in that two different of the image regions (R2, Rn) are coded to the predetermined accuracy levels independently of each other. 521 021 lb 3. Method for transmitting an image (3) between a transmitter (2,5,6) and a receiver (7,8), which. method comprises the steps of: - dividing the image (3) into at least two image regions (R1, R2, Rn); - encoding the image regions (R1, R2, Rn) into an encoded symbol stream (21), the encoding utilizing a symbolic representation and having predetermined accuracy levels in the image regions; and - compressing the coded symbol stream into a compressed bitstream (PS1, 27); characterized in that the method comprises the steps of: - generating (22) a definition (PS2) of a mask (PS2) for at least one of the image regions (R2, Rn), two different of the image regions (R2, Rn) being coded to the predetermined accuracy levels independently; transmitting said definition (PS2) to the receiver (7); - transmitting the compressed bitstream (PS1, 27) to the receiver (7,8); and - decoding (33,34) in the receiver. using the said definition. 4. A method according to claim 1, 2 or 3, characterized in that only predetermined parts of the compressed bitstream (PS1, 27) are decoded. 10 15 20 25 521 021 I? Method according to one of Claims 1, 2, 3 or 4, characterized by the generation of a topology description, which indicates the topological relationship between objects (O1, O2, O3, O4) and shapes (S1, S2, S3, S4) in the image. Method according to one of Claims 1, 2, 3 or 4, characterized by generating a shape description which indicates the appearance of the closed boundary line (Si) of an object (O1, O2, O3, O4) in the image. Method according to one of Claims 1, 2, 3 or 4, characterized by generating a segment description, which indicates which transform coefficients belong to the respective segments. Method according to claim 7, characterized by generating a sub-segment description, which indicates which transform coefficients belong to an independently decodable part of a segment. Method according to any one of claims 5, 6, 7 or 8, characterized by generating a pointer, which indicates a position in the bit stream (27) for the respective of the above-mentioned descriptions. 10 15 20 25 10. An apparatus for transmitting an image (3) comprising: a transmitter (2,5,6) and a receiver (7,8); means (4,5) for dividing the image (3) into at least two image regions (R1, R2, Rn); an encoder (5) for encoding the image regions (R1, R2, Rn) into an encoded symbol stream, the encoder utilizing a symbolic representation and having predetermined accuracy levels in the image regions; a compression device for compressing the encoded symbol stream into a compressed bit stream (PS1, 27); and (2,5,6) (PS1, 27) that said (718): - means in the transmitter for transmitting the compressed bitstream to the receiver characterized in that the device also comprises: ll. means (5) for generating (22) a definition (PS2) of an outer boundary line (Si) for at least one of the image regions (R2, Rn); means in the transmitter (2,5,6) for transmitting said definition (PS2) to the receiver (7,8); and decoder (8) in the receiver for decoding (34,35) the compressed bitstream (PS1,27) by means of said definition (PS2). characterized in that (24) predetermined device according to claim 10, the coding device is arranged to encode two different of the picture regions (R2, Rn) to the accuracy levels independently of each other. An apparatus for transmitting an image (3) comprising: a transmitter (2,5,6) and a receiver (7,8); means (4,5) for dividing the image (3) into at least two image regions (R1, R2, Rn); an encoder (5) for encoding the image regions (R1, R2, Rn) into an encoded symbol stream, the encoder utilizing a symbolic representation 13. and having predetermined accuracy levels in the image regions; a compression device for compressing the encoded symbol stream into a compressed bit stream (PS1, 27); and means in the transmitter (2,5,6) for transmitting said compressed bitstream (PS1, 27) to the receiver (7,8); characterized in that the device also comprises: means (5) for generating (22) a definition (PS2) of a mask (PS2) for at least one of the image regions (R2, Rn), the coding device (5) being arranged to code ( 24) two different image regions (R2, Rn) to the predetermined accuracy levels independently of each other; means in the transmitter (2,5,6) for transmitting said definition (PS2) to the receiver (7,8); and decoder (8) in the receiver for decoding (34,35) the compressed bitstream (PS1, 27) by means of said definition (PS2). Device according to Claim 10, 11 or 12, characterized in that the decoder (8) is arranged to decode only predetermined parts of the compressed bit stream (PS1, 27). 10 15 20 25 14. 15. 16. 17. 18. 521 02 1 '20. Device according to claim 10, 11, 12 or 13, characterized. in that the transmitter (2,5,6) has means for generating a topology description, which indicates the topological relationship between objects (O1, O2, 03, O4) and shapes (S1, S2, S3, S4) in the image. Device according to claim 10, 11, 12 or 13, characterized in that the transmitter (2,5,6) has means for generating a shape description, which indicates the appearance of the closed boundary line (Si) of an object (O1, O2, O3, O4) in the picture. Device according to claim 10, 11, 12 or 13, (2,5,6) which indicates which features. from the fact that the transmitter has means for generating a segment description, transform coefficients belonging to each segment. Device according to claim 16, characterized in that (2,5,6) sub-segment description, the transmitter has means for generating one which indicates which transform coefficients belong to an independently decodable part of a segment. according to claim 14, 15, 16 or 17, (2,5,6) which indicates a position in the device characterized in that the transmitter has means for generating a pointer, the bit stream (27) for the respective of the above-mentioned descriptions.