WO1999048297A1

WO1999048297A1 - Method for increasing the resolution of an image

Info

Publication number: WO1999048297A1
Application number: PCT/EP1999/001859
Authority: WO
Inventors: Winfried Jentsch
Original assignee: Coin Institut Für Computer-Integriertes Engineering Gmbh
Priority date: 1998-03-19
Filing date: 1999-03-19
Publication date: 1999-09-23
Also published as: DE19812118A1; AU3416099A; EP0983690A1

Abstract

The invention relates to a method for increasing the resolution of an image, especially of a TV image. The method comprises the following steps: compiling, digitizing and storing image information to an original image having a given number n1 of pixels in s1 columns and z1 lines. In addition, a data compression inverse transformation is compiled and stored. The data compression inverse transformation is applied on the original image in order to obtain a target image having a number n0 of pixels in s0 columns and z0 lines. The obtained target image having an improved resolution is stored and/or optionally displayed.

Description

- 1 -

Process for increasing the resolution of an image

The invention relates to a method for increasing the resolution of an image, in particular a TV image, according to claim 1.

In the case of known image display devices for displaying images, the image information of which is arranged in rows and columns in matrix form, a large number of different image formats has become common.

Accordingly, there are also very different forms of coding and transmission of the image information such. B. from a unit that generates and provides the images to the image display devices.

In television technology, for. B. the PAL, SECAM, NTSC and the HDTV method for image data coding and / or transmission have become standard.

On the other hand, in the field of computer technology, a variety of display standards for data terminals, e.g. B. in the form of graphic terminals known. In this area of computer graphics and animation, a considerable improvement in quality has been achieved compared to conventional television technology in that a much higher resolution with an increased refresh rate is used in the full-screen process. So z. B. in the SVGA standard, especially in multimedia applications, 1280 x - 2 -

1024 or 1600 x 1200 picture elements or picture pixels with refresh rates of the full pictures of at least 60 Hz are common, whereas with normal television in the field mode one works with 50 or 60 Hz.

HDTV (High Definition Television) tries to bridge the gap between the standards in the computer sector and the television media sector.

The convergence of the two technical directions of television on the one hand and computer applications in the multimedia and graphics sector on the other has led to the creation of effective methods for the digital processing of image information. It is a concern of some of these methods to use the redundancy in the image information in order to compress the amount of image data, which can take on enormous dimensions, to a manageable extent, whereby losses of information should be avoided as far as possible.

On the other hand, it is desirable that common and low-resolution TV pictures as part of the convergence of TV and computer graphics to a multimedia unit (WEB-TV) on high-resolution image output devices, eg. B. a SVGA monitor, can be displayed using the full resolution of the output device.

The object of the invention is to create a method with which a given image with a certain image resolution can be used to produce an image with improved image resolution, in particular with doubled image resolution, particularly simply and quickly. - 3 -

The object is achieved according to the invention with a method for increasing the resolution of an image, in particular a TV image. Advantageous developments of the method according to the invention are the subject of the dependent claims.

The achievement of the object essentially consists in that the image information for an archetype, which contains a given number of image elements, which are arranged in columns and rows in matrix form, is made available, digitized and stored. Providing z. B. can be realized via a conventional TV reception stage or via a conventional video input.

Then the image is decoded according to analog and / or digital transmission and coding methods (PAL, PAL +, SECAM, NTSC, ... and VHS, SVHS, ... or DVB, DVD, DVHS, ...) and if necessary e.g. digitized by means of an A / D converter and then stored in a conventional digital image memory for further processing.

The method according to the invention further provides that a data compression reverse transformation is provided and stored. This can e.g. B. hardware by an appropriate image processing processor or very generally by a microcomputer in which an adaptable image processing algorithm is stored.

Then a so-called target image with a certain number of picture elements, which are also arranged in columns and rows in matrix form, is generated by the data compression return provided and stored. - 4 -

transformation is applied to the archetype or to the image information for the archetype. The data compression reverse transformation can also be understood here as a data decompression operation for a given data compression operation. In general, the decompression of the original image data can be based on any data compression standard such as JPEG, MPEG, MPEG 2 or filters such as wavelets, fractals or the like.

Finally, the image information obtained from the decompression of the original image is stored and, if necessary, displayed.

A core idea of the present invention is therefore to conceive a given archetype, the resolution of which is to be increased, fictitiously as a compressed image and to perform a data decompression or expansion on this fictitiously compressed image via the data compression re-transformation to be applied or the decompression operation, thereby performing an image is generated with increased resolution, d. H. with more picture elements than the archetype.

In principle, the data compression re-transformation in question here can be any common method with which a digitally compressed data record can be decompressed or expanded, with which a previously performed data compression can essentially be undone.

Advantageously, however, data compression transformations are used which work lossless or almost or essentially lossless. Then, from the given data compression transformation, an inverse to this transformation, ie back - 5 -

common data compression reverse transformation can be created. This is then applicable to the original image to create a target image with a higher resolution, i. H. to generate more picture elements compared to the archetype.

The method according to the invention is particularly simple and flexible if the data compression reverse transformation is based on a so-called wavelet transformation and / or a fractal decomposition.

With wavelet transformations, the image information losses when compressing image data are particularly small, or they disappear entirely. Furthermore, it has been shown that the inverse transformation to a wavelet transformation delivers good results even if the difference data describing the difference between the original image and the compressed image is lost or is not used in the inverse transformation. In addition, the wavelet transformation, and consequently also the data compression reverse transformation resulting from a wavelet transformation, can be adapted particularly flexibly to the amount of data to be processed, so that a particularly fast and real-time method for increasing the resolution of an image can be provided .

In the case of the inverse wavelet transformation, as already mentioned above, the difference image data can also be dispensed with.

On the other hand, from the underlying wavelet transformation, the data set for the difference images can subsequently be converted from a sequence of compressed images into the original image - 6 -

iterative application of the wavelet transformation to the original image. Another core idea is that the underlying wavelet transformation, in its effect as a data compression transformation, already contains information about how the application of the inverse wavelet transformation would affect the archetype. This aspect is explained in more detail below.

Each wavelet transformation is based on at least one filter pair consisting of a blurr or low-pass filter on one side and a Sharpen or high-pass filter on the other side.

It is particularly advantageous in the method according to the invention that the data compression reverse transformation is based on at least one blurr or low-pass filter and at least one sharpen or high-pass filter, which are mirrorable or invertible or at least essentially meet these properties. An inverted or mirrored blurr or low-pass filter and an inverted or mirrored Sharpen or high-pass filter are then assigned to the underlying filters.

These filters or filter pairs essentially result from the underlying wavelet transformation, which is adapted and selected in relation to the special circumstances via its wavelet basic functions.

In a preferred exemplary embodiment of the method according to the invention, the application of the data compression reverse transformation to the original image has a method step in which a column of image elements - 7 -

every column of the saved image or archetype is inserted. After inserting the columns, there is an intermediate image with a doubled number of columns, which is also buffered. It is then provided that the mirrored or inverted low-pass filter or Blurr filter and the mirrored or inverted high-pass filter or Sharpen filter be modified using known image processing methods and applied to the columns of this intermediate image which has been buffered. Furthermore, the image information then obtained is also stored and / or possibly displayed.

With this procedure, the picture elements to be inserted can either initially contain no picture information. They are then assigned image information by using the inverted and / or modified filters, taking into account the image information of the neighboring columns of the original image. Or the image information results from the above-described interactive application of the wavelet transformation to the original image to generate a sequence of compressed intermediate images.

This exemplary embodiment of the method according to the invention ensures that a target image which has twice the number of columns as the original image is generated from the imported, digitized and stored original or original image.

At the level of sensory physiology this embodiment makes use of the fact that the horizontal resolution of the human visual perception system is greater than that in the vertical direction. This sense-physiological peculiarity is already used in the PAL + for the 16: 9 format, for example. - 8th -

An objective increase or improvement in the horizontal resolution and texture of an image by a certain value Θ results in an apparent improvement for the viewer on a subjective level, which is even greater than the amount Θ.

In a further exemplary embodiment of the method according to the invention, it is preferably provided that a line with picture elements is inserted behind each line of a buffered image and the image information of the intermediate image thus obtained is also buffered. Thereafter, the mirrored or inverted low-pass filter or Blurr filter and / or the mirrored and / or inverted high-pass or Sharpen filter is also modified using known methods of image processing and applied to the lines of this buffered image, and the image information obtained is stored and / or reported if necessary.

Here, too, the information initially inserted can be set to zero in the form of additional picture lines, ie it cannot contain any information. It is then generated and assigned by using the inverted and / or modified filters, taking into account the image information of the neighboring lines of the original image. However, the image information can also be generated from the above-described procedure of iteratively applying the wavelet transformation method to the original image in order to generate a sequence of compressed images.

In this embodiment, the sensory physiological characteristics of the human visual perception system are also used again. - 9 -

Objectively, the resolution is doubled even if the new intermediate lines are generated with the same resolution (in columns) as the field lines of the original image. This objectively given improvement in resolution is subjectively not perceived by the viewer as such, because the fields (A and B) of the snapshots (A and B) are superposed or superimposed without loss on the overall resolution by the human visual perception apparatus and in the moving parts fixed by the eye interpolated while maintaining the texture sharpness of the fields. On the other hand, generation of intermediate lines with a lower resolution (column-wise) than that of the lines of the original field leads to a deterioration in resolution which is perceived as subjective in the case of the frame, although objectively an improvement in the resolution of the frame compared to the original field can be achieved.

Based on this preferred embodiment, e.g. B. can be achieved that an incoming TV field, the z. B. is originally coded in the PAL method, is converted into a full frame by subjectively doubling lines that maintain the trigger.

If the two last-described preferred exemplary embodiments of the method according to the invention are carried out in succession, namely first a row doubling and then a column and row doubling, it can be achieved that, for. B. an incoming original or field first in a full picture or TV full picture and then in an HDTV picture with double the number of columns and lines, ie with four times the image resolution compared to the full picture. - 10 -

In this way, it is possible to operate known computer monitor arrangements with high resolution using the full-screen or non-interlaced method and to obtain conventional television pictures via a reception input stage and to display them on this monitor with high quality in resolution and picture smoothness.

In this way, a method is created with which both conventional computer graphics applications or animations are possible and, moreover, a high-quality output device is also provided for conventional TV programs.

As already mentioned above, it is advantageous that a compressed image is generated for an original image by applying the data compression transformation, which image can in particular be essentially transformed back into the original image. It is further provided that the original image, the compressed image, the data compression transformation and / or a transformation is generated in real time by the use of suitable image processing operators, which, when applied to the original image, improves the resolution Lead image leads.

A core idea of this procedure is to obtain conclusions about the behavior of the data compression reverse transformation in addition to the image information from the original image and the information from the data compression transformation. The data transformation inverse transformation can be constructed from this in real time. The information in the form of the wavelet coefficients and / or image processing operator coefficients can be applied - 11 -

the data compression transformation to the original image and the image processing operators to the result of the data compression transformation of the original image and the image information obtained therefrom from the compressed image. The conclusions provide information about the application of this reverse transformation to the archetype.

It is advantageous here that the data compression reverse transformation, in particular in real time, also produces a texture-improving data compression reverse transformation which, when applied to the original image, not only leads to an improvement in the resolution of the target image, but also to a corresponding texture improvement.

This procedure means that other image properties can also be changed, in particular when adapting the original image to a high-resolution output device. It is also provided that certain data processing methods such as gray scale scaling, edge thinning or the like are used.

All of these procedures and procedures are also available when processing e.g. Video recordings where no real-time processing is necessary.

If, in the procedure just described, the data compression transformation was applied only once to the original image, a large amount of individual information is preferably obtained by applying the data compression transformation iteratively to the original image and the resulting compressed subsequent images . From the individual information of successive iteration steps, coefficients of image processing operators for the transformation - 12 -

tion of the filters identified (transformation coefficients). Furthermore, it is provided that the information from the original image, the data compression transformation, in particular from the transformation coefficients and the sequence of compressed images, is used to generate and provide the data compression reverse transformation.

In a further preferred exemplary embodiment of the method according to the invention, an extrapolation method is used when generating and providing the data compression reverse transformation from the iterated application of the data compression transformation to the archetype, in which the data just mentioned serve as input data.

In all of these methods and procedures, working with a priori provided wavelet coefficients and parameterized image processing operators is available for generating a resolution-improved image BO from the original image B1 in real time. The parameterization of the image processing operators used for the data compression transformation is calculated on the basis of a representative sample of HDTV-resolved video sequences by using known polyoptimization methods for minimizing the difference between the data compression back-transformations calculated from the HDTV images BO and those according to the invention Methods and procedures generated data compression reverse transformations IW-generated determined, the TT low-pass component calculated by IW corresponds to the original image Bl provided.

The invention is explained in more detail below with the aid of a schematic drawing based on preferred exemplary embodiments. In this shows - 13 -

1 is a block diagram generally showing a data compression method;

2 is a block diagram illustrating the basic procedure for the method according to the invention for increasing the resolution of an image,

3 is a block diagram illustrating a first stage in the use of a wavelet transform for data compression.

4 is a block diagram illustrating the iterated application of the wavelet transform.

5 shows a block diagram which represents an exemplary embodiment of the method according to the invention,

6 shows a block diagram which represents a further exemplary embodiment of the method according to the invention,

7 is a block diagram illustrating an apparatus for carrying out the method according to the invention,

8 is a block diagram illustrating an apparatus for executing the method according to the invention in accordance with a further exemplary embodiment, and

9 is a block diagram illustrating an embodiment of the method according to the invention. - 14 -

1 shows a block diagram which basically represents a data compression method. In this process, an original image or archetype B1 is assumed. This archetype B1 has nl picture elements bl, which are arranged in matrix form in zl rows and sl columns. The picture elements bl themselves contain the actual picture information, e.g. B. in the form of RGB coding, in which the individual portions of the colors red, green and blue are described.

A compression method or a data compression transformation R is applied to this archetype B1. After applying this transformation R, a target image B2 is obtained which has n2 picture elements b2, which in turn are also arranged in matrix form in s2 columns and z2 lines.

When the data compression transformation R is applied to the original image B1 to obtain the compressed image B2, image information Δ may be lost under certain circumstances. In this case one speaks of a so-called lossy data compression R with an image difference Δ. If the image difference from original image B1 to compressed image B2 is zero, then a lossless data compression transformation R is referred to.

FIG. 2 shows a block diagram which schematically represents the principle of the method according to the invention. In this method, the original image Bl received and / or digitized and provided from a data carrier, which has nl picture elements bl in matrix form in zl rows and sl columns, is treated as if it were already the result of the application of a data compression transformation R on a fictitious original picture BO. - 15 -

Under this condition, the inverse or mirrored transformation IR, namely the data compression reverse transformation or data decompression transformation, is now generated and provided and applied to the archetype B1. As a result, the target image BO results in an image with nO picture elements bO, which are arranged in zO rows and so columns.

When the data compression reverse transformation IR is applied to the original image B1, the so-called image difference Δ can either be assumed to be zero, or it is artificially taken into account either in the construction of the target image BO or directly in the construction of the data compression reverse transformation IR.

As a result, after the data compression reverse transformation IR has been applied to the original image B1, a target image BO is obtained which has a higher resolution.

FIG. 3 shows a block diagram in which the simple application of a wavelet transformation W to an archetype B1 with nl picture elements in zl rows and sl columns is shown in matrix form. The basis of the wavelet transformation is a filter pair with a high-pass filter H and a low-pass filter T.

Starting from the original image Bl, the low-pass filter T and the high-pass filter H are first applied to the lines of the original image Bl. The result is two line-by-line low-pass or high-pass filtered intermediate images 11 and 12. From these intermediate images, every second line is first removed from the intermediate images 11 and 12 using a method Z for line removal. As a result, intermediate images 21 and 22 are obtained. - 16 -

The low-pass filter and the high-pass filter are now reapplied to these intermediate images 21 and 22. As a result, four intermediate images 31, 32, 33 and 34 are obtained, which are additionally low-pass and high-pass filtered in columns. Every second column of these intermediate images is now removed from these intermediate images 31, 32, 33 and 34 by means of an operator S for column removal. The result is four intermediate images TT, HT, TH and HH.

After applying the wavelet transformation W once, four individual images TT, HT, TH and HH are obtained. The portion TT is a low-pass line, low-pass column image, in which every second column and every second line is deleted compared to the original image B1. It is therefore a reduced and smoothed version of the original or original image B1 and is referred to as B2.

The portion TH is a low-pass row high-pass column image and contains difference information between the original image Bl and the portion TT in the horizontal direction of the image information.

The portion HT is a high-pass line low-pass column image and contains the image difference information for the original image B1 in the vertical direction.

Finally, the portion HH is a high-pass line-high-pass column image and contains the difference information with respect to the original image B1 in the diagonal direction.

When the original image Bl is reconstructed from the parts TT, HT, TH and HH, the individual operations of removing columns and rows are made by inserting columns or rows - 17 -

len, which consist only of zeros, are undone, and the filter operations are canceled with mirrored or inverted filters. By superposition or superposition of the parts TT, HT, TH and HH, the image information of the original image B1 is then obtained. Optionally, all high-pass or H components, i.e. H. HT, TH and HH are neglected or set to zero.

FIG. 4 shows a block diagram in which the iterated application of the wavelet transformation W is shown.

Starting from an archetype B1, the wavelet transformation W is first applied to the archetype B1 itself. The result is a compressed image B2, which corresponds to the portion TT, and high-pass portions (HT, TH, HH), which carry the difference information to the original image B1.

Then the wavelet transformation W is applied to the compressed image B2 of the first compression stage from the TT component of the first application of the wavelet transformation W. The result is a further compressed image B3, namely as a TT component of the second iteration stage of the wavelet transformation W. The difference information from image B3 compared to image B2 is contained in new high-pass components of the second stage (HT, TH, HH).

Further iterated applications of the wavelet transformation W on the respective TT components result in further successors, i. H. further compressed drawing files, B4, B5, with corresponding high-pass components.

In the iterated application of the wavelet transformation W, the individual iterations are reversible and the inverse - 18 -

gen IW lead back to the respective previous image and finally also to the original image B1 if, as described above, all operations are reversed and the difference information from the high-pass components is used.

5 schematically shows in a block diagram the sequence of an exemplary embodiment of the method according to the invention, in which a PAL input signal forms an original picture B1, namely a TV field, by using the data compression reverse transformation in the sense of an inverted wavelet transformation IW Full picture BO with double the number of lines compared to the original picture Bl is generated.

The inverted wavelet transformation IW, which results from the wavelet transformation W, can already be provided and provided. However, as is shown in this exemplary embodiment in FIG. 5, the inverted wavelet transformation IW can also be derived by generating an image B2 that is compressed with respect to the original image B1. In this case, the compressed image B2 is obtained by applying the given wavelet transformation W to the original image B1 as has been described in general above. Under the premise that the compressed image B2 must be able to be transformed back into the original image B1 by means of the inverse wavelet transformation IW, the inverted wavelet transformation IW can then be determined in real time and applied to the original image B1 for the reconstruction of the full image BO.

The image information of the original image B1, the first compressed image B2 and the information of the wavelet transformation W, in particular the results of the image processing of the wavelet transformation W, that is to say, for example, the wavelet coefficients, go directly into the construction and generation of the inverse wavelet Transformation IW a. - 19 -

FIG. 6 shows a further exemplary embodiment of the method according to the invention, again using an original image B1. This archetype B1 is also converted into a target image BO with improved resolution by means of the inverse wavelet transformation IW, which originates from a wavelet transformation W, with certain image parameters, e.g. B. the texture of the image can be changed, adjusted and improved.

As in the embodiment of FIG. 5, the given wavelet transformation W is also applied to the original image B1 in order to obtain a compressed image B2. This compressed image B2 represents the TT portion of the first iteration level of the wavelet transformation W on the original image B1. The difference information is contained in the portions HT, TH and HH of the first iteration level.

In contrast to the exemplary embodiment in FIG. 5, iteration, i. H. in the application of the wavelet transform W, continued. This means that the wavelet transform W is now on the TT portion of the first iteration stage, i. H. is applied to the compressed image B2. As a result, the next follow-up element, that is to say a further compressed image B3, is obtained, along with the portions HT, TH and HH describing the image difference of the second iteration stage of the application of the wavelet transformation W. As in the exemplary embodiment in FIG. 5, so too the condition that the predecessor image B2 must be reconstructable from the compressed image B3, that is to say the TT component of the second iteration stage, by applying the inverse wavelet transformation IW to the image B3.

After a given number of successive applications of the wavelet transformation to the sequence of compressed images B2, B3, ... all information, - 20 -

namely the original image B1, the compressed images B2, B3, ... and the difference image components (HT, TH, HH) of the individual iteration stages, in particular combined and / or evaluated in an extrapolation process E by parameterized image processing operators, and an inverse wavelet A transformation is obtained therefrom, which in the displayed manner produces a corresponding target image BO with improved resolution and possibly adapted graphic parameters, such as texture and the like, from the original image B1.

To speed up the implementation of the exemplary embodiments of the method according to the invention, all method steps can also be applied to a local point environment of each point of the original image BO. This procedure results in the possibility of generating the target image BO locally and in real time from the input or original image B1. Due to the property of the locality of the processing methods, a particularly effective because of simplified and accelerated processing can be achieved, so that z. B. also offers real-time processing of a stream of TV signals entering a receiver.

Accordingly, it is possible, without or with a negligible time delay, to read in, digitize, and transform TV programs, in particular moving image sequences, on a picture display device on which one can display in computer picture quality, and to transform them to a standard with higher graphic resolution and improved picture parameters .

The local image processing of a point of the original image BO must of course also the point environment of the respective point - 21 -

to include. This point environment can consist of the closest neighbors. However, other extended point environments, e.g. B. with neighbors after next, be provided.

In an extreme case, the entire archetype BO is analyzed in order to improve the point environment of a point of the archetype BO in its resolution.

Fig. 7 shows the block diagram for the basic structure of a device for performing the method according to the invention.

An original image B1 to be processed is received or made available via an image receiving / providing device BE. This can be a TV input stage, a receiving device for video images or the like. It is also contemplated that digitally existing image data can be recorded in the image receiving / providing device.

The received and provided image data for the original image B1 are then forwarded to an original image memory US and stored there digitally. If the image data for the archetype B1 is not already available digitally, it is provided to digitize this data using an analog / digital converter AD.

The original image memory US is connected to an image processing device BV. The image processing device has an analysis section AN, which is provided with a plurality of image memories S1 to SN. Furthermore, a transformation unit T is provided which - 22 -

Formations of the original image Bl from the original image memory US, processed to a target image BO and this image information of the target image in digital form and optionally after appropriate image manipulation to an output memory AS.

The output memory AS can be a fast video image removable memory, which in turn outputs data to an image output device BA. The image output device BA can be designed in the form of a TV or video output stage.

In the image processing device BV, the original image B1 is recorded from the original image memory US, and z. B. in the analysis unit AN the iterative applications of a wavelet transformation W are carried out, the individual results of the sequence of compressed images B1 to BN being stored in the memory devices S1 to SN. Furthermore, the wavelet transformation IW inverted to the wavelet transformation W is generated in the analysis unit AN, e.g. an extrapolation-capable inverse wavelet transformation IW is generated by image processing operators. The inverse wavelet transformation IW is then transmitted to the transformation unit T.

After the inverse transformation IW has been obtained, the transformation image T reads out the original image B1 from the original image memory US and processes it further into a resolution-improved target image BO.

For the local generation of resolution and / or texture-improved images in real time and in particular for the construction of the inverse wavelet transformation IW, the wavelet method is used as follows: - 23 -

First, e.g. B. received the transmitted sequence of TV fields or TV frames as usual and digitized in real time. After being digitized, each received TV picture is written as a TT part in a buffer. The high-pass and low-pass filters are then applied to the buffered image, and the images obtained are in turn stored as portions TH, HT and HH.

After that, there are several ways to proceed. For the local generation of full images from fields in real time, every second column is then removed from the temporarily stored portions TH, HT and HH. The results are again stored as TH, HT and HH portions in buffers.

Then the contents of the buffer stores are processed with image processing operators, e.g. Operators for edge sharpening, line thinning, image zooming and / or for gray value scaling - especially after previous zoom, color decomposition and / or gray value scaling of the pixels in yuv and / or RGB coding - processed, also to e.g. To achieve surrealistic, naturalistic or individually adjusted or optimized contrasts, colors and / or textures in the target image. From the information of the intermediate value memories for the parts TT, HT, TH and HH, the intermediate lines for generating the full picture from the field are then generated in reverse transformation, the high-pass parts TH, HT and HH also being made zero in order to simplify and accelerate the process can be placed.

Instead of removing columns or rows, the local generation resolution and / or texture-improved images can be dispensed with in real time. - 24 -

Then z. B. generated from the information of the parts TT, HT, TH and HH intermediate lines and intermediate columns for a high-resolution full picture in the HDTV standard after the corresponding image processing operators were applied to the parts.

On the other hand, further iterations of wavelet compression can also be carried out on the components HT, TH and HH. The respective proportions HT, TH and HH are then iterated as standard with the wavelet transformation and then appropriately superposed for the zooming, edge thinning or the like after the use of image processing operators. The result of the superposition is determined using suitable image processing methods, e.g. B. the generation of a three-dimensional spline surface and described with a fitting function F over raster planes. Finally, in an extrapolation process on the basis of the fitting function and the proportions TH, HT and HH, the back transformation to the full HDTV image is achieved as the target image with quadrupled resolution and improved texture.

For the purposes of the invention, the term texture is also intended to mean in particular the gray value and / or color contrast. Accordingly, the term texture improvement is also understood as an improvement in the gray value and / or color contrast in the reproduction of the gray value and / or contrast quality expressions contained in the original image.

8 schematically shows in the form of a block diagram a device 1 with which the method according to the invention for increasing the resolution of a raster image can be carried out. - 25 -

Via an analog input stage 80, which e.g. consists of a conventional TV tuner and which different types of TV pictures in PAL, NTSC, SECAM, DVB, SVHS or DVD process can receive and evaluate via an input and / or receiving device, corresponding image data streams in the Device 1 fed. The corresponding color coding is assigned to the received images via a color or color decoder 81.

In general, after reception and color decoding, a digitization step takes place via one or more A / D converter devices 84, which can be omitted if the input stage 80 is designed for digital signals. The result of the analog / digital conversion, and in particular the digital color signals, are stored in one or more intermediate buffers 85. This can be a memory area 85, which is designed to record one or more digital raster images and / or to release them in rapid alternation.

In an inverse wavelet encoder 82, the data compression re-transformation method is determined via the access to the image buffer memory 85 and / or is carried out on the buffered digital images. Then the generated, i.e. Images with improved resolution and texture are processed in a display processor device 83 and then output either directly or via a downstream digital / analog conversion device 87 to a display device 88 for visual display.

All operations of all components are regulated and controlled by a higher-level control processor 86. - 26 -

With reference to FIG. 9, an embodiment of the method according to the invention is shown in the form of a block diagram, in particular with regard to the generation or generation of a data compression reverse transformation.

Generation from the archetype

The following section deals with the description of the offline determination of parameters of the image processing operations for the generation of the data compression reverse transformation from the original image.

A core idea of the procedure when using image processing operations to generate a resolution and texture-improving data compression reverse transformation is the use of a parameter identification method for determining a parameterization Pi of an image processing operation BOi by minimizing the difference between one of the original image and the use of a wavelet method WV generated data compression transformation DT1 of the first stage, where DT1 = (TT1, TH1, HT1, HH1), and a data compression transformation DT2 generated from the repeated application of the wavelet method WV on the TTl image, where DT2 = (TT2, TH2, HT2, HH2).

The parameters of the parameterization Pi of the image processing operations BOi and the wavelet transformations WV are determined in such a way that a data compression reverse transformation DRT1 arises when using or possibly a sequence of applications from B0i (Pi, 2) to DT2, DRT1 = (TT1 *, TH1 *, HT1 *, HH1 *), which with DT1 are essentially - 27 -

lie or largely agrees that the original image is restored by applying DRT1 to TT1 from DT1 essentially or almost lossless. The parameterization Pi of the image processing operation BOi is essentially or largely independent of the process stage n that generates it, i.e. with the following restriction: B0i (Pi, l) = B0i (Pi, 2).

In the case of an application or possibly a sequence of applications from BOi (Pi, l) to DT1, a data compression reverse transformation DRTO then arises, where DRT0 = (TT0 *, TH0 *, HT0 *, HH0 *), which is associated with a fictitious DTO to the extent that it essentially or largely coincides with the fact that by using DRT1 on the original image or on TTO *, a substantially or almost four times better resolved image is generated in HDTV quality.

Another core idea of the invention is that the parameterization of the image processing operations BOi (Pi, n) is generated in an optimization process carried out offline for the purpose of permanent use in the real-time generation of resolution and texture improvement of the original image sequence. For the determination of the parameterization Pi, it is preferable to use a representative sample of HDTV-resolved images BOj and the data compression transformations DTOj with DT0j = (TOj, THOj, HTOj, HH0) generated by using a WAVELET method WV. to work at the zero-th stage. In the TTOj portions, the data compression transformations of the HDTV pictures BOj correspond to the original pictures Blj, which are available as TV-resolved pictures as standard. The determination of suitable image processing operations (a), the order of their application

(b) and the extrapolation of optimized parameterizations

(c) is carried out offline using parameter identification techniques in advance of an effective technical implementation of the online process. - 28 -

Among the known image processing operations, the following types are particularly suitable for the generation of the data compression reverse transformation DRTO from the data compression transformation DT1 or iterative data compression transformations DT1, DT2, DT3, ..., which are now listed and illustrated in detail become:

(a) Image processing operations

Zooming

The pixel patterns of the H components of DTn, with DTn = (TTn, THn, HTn, HHn), the nth application of the data compression operation or transformation over the original image are achieved by doubling the successively performed on the DTn grids Rows and columns brought to four times the grid areas in the DRTn-1.

The pixel patterns of the H components of DTn, with DTn = (TTn, THn, HTn, HHn), the nth application of the data compression operation over the original image, are fourfold by empty lines and empty columns inserted in the DTn grid Grid areas in DRTn-1 pulled apart.

Antialiasing

The saw teeth in the grid areas for DRTn-1, which result when the above zoom operator is applied to the pixel pattern of the H components of DTn, with DTn = (TTn, THn, HTn, HHn), are reduced by interpolative reduction of the gray values on the Edges eliminated.

Applying the above zoom operator to the pixel pattern of the H components of DTn, with DTn = (TTn, THn, HTn, HHn), in - 29 -

The gaps in the grid areas for DRTn-1 are smoothed by adding gray pixels interpolated between adjacent pixels via empty lines or columns.

Spline adjustment / fitting

Given are the raster images TTn-1, THn-1 and HHn-1 of the data compression transformation DTn and the raster images TTn, THn, HTn and HHn of the data compression transformation DTn. The intensity value distribution for the individual colors of the RGB signal (or the y value of a yuv-coded raster image or other color coding) can be described above the raster level by a three-dimensional surface, which is an adjustment or a fit of the gray value intensity distribution by a surface represents in space. Spline techniques can be used to adjust or fit the gray value distribution through n-dimensional areas above the image grid area, which can simultaneously achieve image smoothing by reducing pixel noise. With integral treatment of the three colors of the RGB signal, n = 5, with separate treatment of red, green and blue, n = 3. When working with yuv-coded signals, you can work efficiently with the y-value alone.

For the raster images TTn-1, THn-1, HTn-1 and HHn-1 from DTn-1 as well as for the raster images TT * nl, TH * nl, HT * nl and HH * nl from DRTn-1 - generated by executing image processing operations BOi over the raster images TTn, THn, HTn, HHn from DTn, such as zooming to the image format of DTn-1 - the fitting surfaces F (XYn-l) or F * (XYn-l) as 2D Splines determined, where X and Y are a low pass and a high pass, respectively. By using a method for - 30 -

Difference minimization over individual or all surface pairs F (XYn-1) and F * (XYn-1) to DTn-1 and DRTn-1 an optimal parameterization Pi for the image processing operation B0i (Pi, n) is identified, which when applied to DTn a Data compression reverse transformation DRTn-1 is produced, which essentially corresponds to DTn-1 in that the image Bn-2 is equally well restored by applying DTn-1 as well as DRTn-1 to the image Bn-1.

The constraints of an invariance of the gray value scaling with respect to the fitting areas F (XYn-l) or F * (XYn-l), which underlies the different optimization for the identification of the parameterization Pi, ensures that the absolute amounts of the intensity fitting areas over the Gray values described on the raster level remain invariant compared to the image processing operation BOi, but shift the locations of the gray values in line and edge thinning and smoothing in the raster plane.

The parameter identification by minimizing the difference between F (XYn-l) and F * (XYn-l) under the condition of preserving the volume integral and scaling the gray value distribution is the main reason for a texture improvement when generating a resolution increase of the original image by inverse data compression to watch.

Edge thinning / thickening

The pixel patterns created when using the zoom operator on DTn in the DRTn-1 raster areas are to be thinned such that for the total number of pixels S (sum of all not - 31 -

zero-valued pixels) that apply between the data transformation operation DTn and the data reverse transformation operation DRTn-1 generated therefrom: S (DRTn-1) = 4 * S (DTn), with DTn = (TTn, THn, HTn, HHn ).

The pixel patterns created when the zoom operator is used on DTn in the DRTn-1 raster areas are to be thickened so that for the total number of pixels S (sum of all non-zero-value pixels) between the data transformation operation DTn and the one resulting therefrom generated data reverse transformation operation DRTn-1 applies: S (DRTn-1) = 4 * S (DTn), with DTn = (TTn, THn, HTn, HHn).

Anti-aliasing

When using the zoom operator on the pixel pattern of the H components of DTn, with DTn = (TTn, THn, HTn, HHn), the gaps in the edges that arise in the DRTn-1 grid areas are to be closed in such a way that for the total number of pixels S (sum of all non-zero-valued pixels) between the data transformation operation DTn and the data re-transformation operation DRTn-1 generated therefrom applies: S (DRTn-1) = 4 * S (DTn), with DTn = (TTn, THn, HTn, HHn).

The edge profile gaps in the DRTn-1 grid areas which result when the above antialiasing operator is applied to the pixel patterns of the H components of DTn, with DTn = (TTn, THn, HTn, HHn) are shown by edge profile pixels adjacent via empty lines or empty columns added that for the total number of pixels S (sum of all non-zero-valued pixels) between the data transformation operation DTn and the data inverse transformation operation DRTn-1 generated therefrom: S (DRTn-1) = 4 * S (DTn), with DTn = (TTn, THn, HTn, HHn). - 32 -

Grayscale scaling

The gray value sums S (TTn), S (THn), S (HTn), S (HHn) are above the intensity distribution of the gray values of the data compression transformation DTn, DTn = (TTn, THn, HTn, HHn), e.g. the y value of the yuv signal or the individual color intensity values of the RGB signal is determined. The quotient of S (XYn) of the nth wavelet transformation divided by S (XYn-l) of the nlth wavelet transformation is QXY, QXY = S (XYn) / S (XYn-l), where X and Y a low pass or a high pass. The gray value sums S * (TT * n), S * (TH * n), S * (HT * n), S * (HH * n) are above the intensity distribution of the gray values of the data compression reverse transformation DRTn after the application of an image processing operation BOi determined on the result DTn of a wavelet transformation of the nth stage.

The gray value sums can e.g. for the y values of yuv-coded pixel grids in the three-dimensional intensity space or for the red, green and blue values in the five-dimensional color intensity space.

When the image processing operators are applied to the pixel pattern of the H components of DTn, with DTn = (TTn, THn, HTn, HHn), the gray values that arise in the DRTn-1 raster areas are to be scaled such that for the integral gray value ( Sum over all pixels) between the data transformation operation DTn and the data re-transformation operation DRTn-1 generated therefrom is achieved that: S * (XYn-1) = QXY * S (XYn). The invariance of the gray value sums and the scaling of the gray value distributions with respect to S * (XYn-l) 1) and A (XYn-l) above the raster level XYn must be observed as a secondary condition. - 33 -

Generation of 3D spline approximations

The grayscale patterns in DRTn-1 that result when the image processing operators are applied to the pixel patterns of DTn, with DTn = (TTn, THn, HTn, HHn), are to be adapted or fitted using 3D spline surfaces F (DRTn-1). Likewise, the pixel patterns of DTn-1, with DTn-l = (TTn-1, THn-l.HTn-1, HHn-1), are to be adapted using 3D spline surfaces F (DTn-l). The parameterization of the spline functions F (DRTn-l) can be identified by minimizing the difference between F (DRTnl) and F (DTn-l). As a secondary condition, the parameter identification states that the following applies to the volume integrals V over the spline surfaces: V (F (DRTn-1)) = V (F (DTn-1)). The spline functions and volume integrals formed over them can e.g. are determined for the y values of yuv-coded pixel grids in the three-dimensional intensity space or for the red, green and blue values in the five-dimensional color intensity space.

The invariance of the volume integrals and the scaling of the gray value distributions with respect to V (F (DRTn-l)) and V (F (DTn-l)) over the respective raster levels XYn-1 must be observed as a secondary condition.

(b) Sequence of image processing operations

The following parameterized image processing operations BOi (Pi, l) from section (a) above are carried out successively, in parallel, locally and / or globally:

1. The data compression DT1 with DT1 = (TT1, THI, HT1, HH1) is generated from the (PAL) original image using a wavelet method. - 34 -

2. A preliminary stage DRTO # of DRTO is generated from DTl by zooming.

3. From DRTO # an anti-aliasing and / or edge thinning / thickening / smoothing is used to generate a preliminary stage DRT0 ## of DRTO which is improved compared to DRT0 #

4. DRTO is generated from DRTO ## by gray value scaling and / or spline adjustment.

5. By applying the data reverse transformation DRTO generated by steps 1 to 4 to the original image or to the TTO * portion from DRTO, a target image that is four times higher resolution and better textured is generated, which corresponds to an HDTV original

(c) Extrapolation of the parameterization of the image processing operation

Data compression methods can be applied iteratively. In the case of iterative application, the back transformation provides the poorer images, the more lossy the data compression operation works. The wavelet method can be carried out lossless if nothing is forgotten from the TTn, HTn, THn, HHn parts of the data compression operations DTn.

The method for generating raster areas DRTn-1 from the DTn rasters can be carried out analogously over more than two data compression stages, so that a sequence of parameter sets Pi, n with n = 1, 2, ... for methods for the extrapolative generation of the Data reverse transformation operation DRTO is available. - 35 -

By separately applying the difference minimization between the data compression operation DTn present for stage n and the grids DTn + 1 of the subsequent data compression operation, parameterization Pi, n + 1 for an image processing operation BOi (Pi, n + l ) determined via DTn + 1, which generates a data reverse transformation operation DRTn such that at least approximately DTn = DRTn applies to DRTn from the application of BOi (Pi, n + 1) to DTn + 1. The respective parameterizations Pi, n + 1 are determined using a suitably selected set of representative archetypes.

An improvement in the parameterization Pi for generating DRTO from DTl is obtained by weighted averaging of the parameterizations Pi, I, Pi, 2, Pi, 3, ...

A further improvement of the parameter identification is by using the difference minimization method over the whole sequence DRT0 = (BOi (Pi, 1 (DT1)), BOi (Pi, l (BOi (Pi, 2 (DT2)))), B0i (Pi, l (B0i (Pi, 2 (B0i (Pi, 3 (DT3)))))), ...) from generated data re-transformation operations DRTn = BOi (Pi, n + l (DTn + l)) with n = l, 2,3, ... reachable.

The data compression reverse transformation DRTO is determined recursively, starting with the result of the last iterative application of the wavelet operation, the parameterizations of the image processing operations being determined such that the result of the data compression reverse transformation DRTn with the previous data compression transformation DTn results in a particularly good fit or a best fit. This ensures that when using the data compression reverse transformation DRTn generated from DTn + 1, approximately the same decompression - 36 -

quality is present, as when using the calculated data compression transformation DTn. Extrapolation of the DRTn generated from the DTn-1 ultimately generates a data compression reverse transformation which, when applied to the original image B1, produces an image BO with improved resolution and texture.

The online generation of the data compression reverse transformation DRTO from the result of the data compression operation DT1 is carried out using BOi (Pi, l) according to the procedure given in section (b).

Generation from apriori information

The online improvement of the parameterization of the image processing operations for the generation of the data compression reverse transformation from aprioni information is explained below.

Another core idea of the invention is the transmission-side coding of elements of a data compression operation, which are sent packaged in such picture lines that are not decoded during the visualization of PAL on conventional TV sets. Similar to the PAL + method, a special decoder is required at the receiving end, which unpacks this information in real time and appropriately calculates it in the coding of the picture.

If the procedure described above for generating a data compression reverse transformation DRTO instead of the PAL original image from the HDTV original image available on the transmission side as data compression operation of the first stage DT1, with DT1 = (TT1, TH1, HT1, HH1), DRT1 = DT1 and TT1 = PAL on the transmission side - 37 -

If the data compression DT2 is calculated from the second application of the wavelet data compression operation on the PAL image (TTl filter from DTl), then the parameterization B0i (Pi, 2) for the receiving-side online Generation of DRT1 from DT2 instead of just extrapolative now in exact form. If this parameterization B0i (Pi, 2) is now also sent in addition to the PAL signal, a receiver-side decoder can do this to the local HDTV generator as parameterization B0i (Pi, l) for calculating DRTO from the receiving side to that from the PAL Provide the calculated DTl. DRTO applied to the PAL picture then results in an HDTV picture.

A further core idea of the invention consists of the data re-transformation operation DRTO = DTO with DT0 = (TTO, THO, HTO, HHO) generated by the transmitter from the HDTV master image, portions of HHO as HHO *, possibly also of THO and / or to encode HTO as THO and / or HTO elements in invisible PAL or PAL + lines. A decoder on the receiving side provides the local HDTV generator with these HHO, THO and / or HTO elements as DRTO for the calculation of HDTV from the PAL picture.

FIG. 9 describes an exemplary embodiment of the method according to the invention for increasing the resolution of a raster image.

This exemplary embodiment for method 2 is based on a TV picture 90 which is transmitted by a transmitter and is equipped, for example, as a PAL TV picture with, for example, 720 x 576 picture elements. This image 90 is then stored in an image buffer 93 on the one hand. On the other hand, the image is processed in a first data compression stage by means of a wavelet transformer 91. The result of this processing stage is in turn stored in an image buffer 94. - 38 -

Due to the boundary conditions described in the section above with regard to the construction of a data compression reverse transformation, the inverse wavelet transformation via the high-pass components TH2, HT2 and HH2 or TH2 *, HT2 *, HH2 * for one is achieved by means of a scaling module 92 and an image converter 95 Image processing operation BOi with a parameterization Pi, 2 is generated and made available to the so-called inverse wavelet transformer 96, which in turn then accesses the image buffer 93 with the PAL-TV image 90 contained therein in order to perform the inverse wavelet transformation to an HDTV picture with eg 1440 x 1052 picture elements.

This then generated HDTV image is then stored via a so-called RAMDAC unit 97 and, if necessary, is transformed analogously, in order then to be shown on a display unit 98.

Claims

- 39 -

claims

1. A method for increasing the resolution of a raster image (B1) which consists of individual image elements which are arranged in columns and rows, in particular a TV image, for output on an image display device, with the steps:

Provision, digitization and storage of image information as an original image (B1) with a given number nl of image elements in sl columns and zl lines,

- Provision and storage of a data compression reverse transformation (IR),

Generating a target image (BO) with a number of nO picture elements in so columns and zO rows by applying the data compression reverse transformation (IR) to the original image (B1),

- Storage and display of the image information obtained as a target image (BO).

2. The method according to claim 1, characterized in that the data compression reverse transformation (IR) is provided to an essentially invertible data compression transformation (R).

3. The method according to any one of the preceding claims, characterized in that the data compression reverse transformation (IR) is based on a wavelet transformation (W). - 40 -

Method according to one of the preceding claims, characterized in that the data compression reverse transformation (IR) is based on at least one low-pass filter (T) and at least one high-pass filter (H), which are essentially mirrorable or invertible and which are each an inverted or mirrored low-pass filter (IT) or high-pass filter (IH) is assigned.

Method according to Claim 4, characterized in that the application of the data compression reverse transformation (IR) comprises the steps:

- inserting a column with picture elements behind each column of a saved picture,

Temporarily storing the image information obtained,

- Apply the mirrored or inverted low-pass filter (IT) to the columns of the buffered image and

- Save the image information.

Method according to one of claims 4 or 5, characterized in that the application of the data compression reverse transformation (IR1) comprises the steps:

Inserting a line with picture elements behind each line of a stored picture,

Temporarily storing the image information obtained,

- Apply the mirrored or inverted low-pass filter (IT) to the lines of the buffered image and

- Save the image information. - 41 -

7. The method according to any one of the preceding claims, characterized in that a compressed image (B2) is generated for an original image (Bl) by applying the data compression transformation (R), which can be re-transformed into the original image (Bl), and that the information from the original image (B1), the compressed image (B2) and the data compression transformation (R) is used in real time to produce the data compression reverse transformation (IR) into a resolution-improved target image (BO).

8. The method according to any one of the preceding claims, characterized in that from the data compression reverse transformation (IR) a texture-improving data compression reverse transformation (TIR), in particular in real time, is generated, which when applied to the archetype (B1) to one Target image (BO) with improved resolution and texture leads.

9. The method according to any one of the preceding claims, characterized in that the data compression transformation (R) is applied iteratively to the original image (B1) in order to obtain a sequence of compressed images (B2, B3 ...), and that the information of the original image (B1), the data compression transformation and the sequence of compressed images (B2, B3 ...) is used to generate and provide the data compression reverse transformation (IR).

10. The method according to claim 9, characterized in that an extrapolation method is used when generating and providing the data compression reverse transformation (IR).