IMAGE ENHANCEMENT SYSTEM
FIELD OF THE INVENTION
The present invention relates to scan line doublers for increasing the number of apparent scan lines of a display device to reduce the visibility of the scan line structure of the picture image. More particularly, the present invention relates to a television, computer monitor or video projector scan line doubter which includes a method that overcomes the limitations of pixel interpolation by scan line averaging. BACKGROUND OF THE INVENTION When all other sources of error and distortion have been removed or minimised by correction or compensation, standard PAL or NTSC colour video images become limited in quality by perceptibility of the line scan structure.
Subjective visibility of the line scan structure is a direct consequence of the limited number of horizontal scan lines in the standard composite picture, and is further a direct consequence of the field by field interlace. A conventional PAL television frame at 25HZ repetition rate is composed of two fields eg F0 and F1. Each field includes 312.5 scan lines, each of which are separated by an unilluminated strip or band. Successive fields are offset so that the scan lines of the next field occupy the unilluminated strips of the present field. This arrangement is followed to minimize perception of 25Hz flicker in the resultant image.
The need to increase the number of scan lines is particularly evident in the application of Field Sequential 3D. In this instance, odd lines of the video image are used to carry the left eye image and even lines the right eye image. Thus, after de-multiplexing, the image intended for each eye is at half the resolution of the original video standard.
One approach to reducing the visibility of the line scan structure of the image calls for estimating, or interpolating, picture elements of additional scan lines from the picture elements already present in the picture image scanned in the conventional format. This prior approach is known in the art as "scan line doubling" or "line doubling", and calls for doubling the number of scan lines from 312.5 to 625 lines per field. Thus 625 lines are presented each 50th of a second.
One prior approach to pixel interpolation is carried out by an intra-field or spatial domain process. The pixel for the unilluminated band between two scan lines is derived as the average of the pixel amplitude and hue of the pixel in the scan line directly above and of the pixel in the scan line directly below. The main drawback of this approach is the reduced resolution or softness of the resultant picture in the vertical dimension at edges and some perceptible 25Hz vertical flicker in the instance of sharp vertical transitions within the picture image. OBJECTIVES OF THE INVENTION
A general objective of the present invention is to provide an improved method and apparatus for television scan line doubling and display. The invention overcomes a number of limitations of the line averaging techniques of prior art and may be simply implemented in readily available hardware or software.
A more specific objective of the invention is to include a method whereby the additional pixel amplitude and hue may be determined via mathematical calculation or a lookup table and applied based upon specific characteristics of the overall image.
With the above objectives in mind, the present invention provides a method of determining an interpreted line in a line doubling system including the steps of: determining the amplitude and hue of pixels on adjacent lines; determining the Root Mean Square (RMS) value of the amplitude and hue of the pixels on adjacent lines; utilizing the RMS value to create said interpreted line. The Root mean Square value may be calculated in hardware or software for each set of adjacent pixels. Alternatively, a lookup table could be used to approximate the Root Mean Square value.
These and other objects, aspects, advantages and features of the present invention will be more fully understood and appreciated upon consideration of the following detailed description of a preferred embodiment, presented in conjunction with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 depicts a segment of a video image.
Figure 2 depicts two tables showing average of two pixels, A and B, and the RMS of A and B. Figure 3 depicts a hardware implementation of the invention.
Figure 4 depicts a simplified hardware implementation of the invention.
Figure 5 depicts the difference between two values A and B.
Figure 6 depicts a lookup table based upon the difference between A and B. Figure 7 depicts a Pseudo RMS value of A and B.
Figure 8 depicts the true RMS value of A and B DETAILED DESCRIPTION OF A PREFERRED EMBODIMENTS
Figure 1 depicts a segment of a video image consisting of Field Zero line zero (F0), Field One line one (F1) and interpreted lines L0 and L1. The individual pixels per line are indicated such that the first pixel on line one of Field 0 is marked PF0,1 the second pixel PF0,2 and the nth pixel PF0,n. Similar terminology is used throughout the figure.
The prior art has described one method of adding additional lines to an interlaced image by inserting an additional pixel having a value equal to the average of the value of the pixel in the line above and the pixel in the line below i.e. if the pixel above has a value A and the pixel below has a value B then the additional pixel will have a value equal to:
New pixel = (A+BV2
The main shortcoming of this technique is that the resultant image tends to look softer or slightly out of focus. This is direct limitation of using a simple averaging technique. For example, if we consider a black and white image as the input source and the above line to be peak white and the below line to be black then the interpolated line will be grey.
In order to overcome this limitation it is considered desirable to insert an interpolated line containing pixels that are more closely associated with the luminance value of the brighter of the two pixels A and B.
In a preferred embodiment this invention discloses a new technique to
achieve this by taking the Root Mean Square (RMS) of the values of A and B rather than the average.
This is illustrated in figure 2 which contains two tables. The first table depicts A along the X axis and B along the Y axis and the average of A and B at the intersection.
The second table depicts A along the X axis and B along the Y axis and the RMS value of A and B at the intersection. The RMS value is calculated from:
RMS Value = (A2+ B 2)/2)
By comparing the Average and RMS tables it is evident that: 1. When A = B, both mean and RMS process yield the same result;
2. When A > B or B > A then the result is squewed closer to the larger value, which is the desired result.
This RMS processing may be implemented in either hardware or software.
A means of implementing the process in hardware is illustrated in figure 3. A pixel from a line in field FO and a corresponding pixel from the line below in F1 are both passed simultaneously to squarer circuits. The output from each squarer is added and this result subsequently divided by two. This intermediate result has its square root taken and the resulting value becomes the new pixel.
If the image to be line doubled is in colour then the original image may well be in RGB format. If this is the case then each of the individual R, G and B values will require to be processed using the RMS method. Such a hardware implementation will require six squaring circuits and three square root circuits.
Both the squaring and square root functions are comparatively difficult to implement in hardware. It is therefore another objective of this invention to disclose an alternative preferred embodiment that enables an RMS value to be calculated in an efficient and effective manner.
Figure 4 discloses an alternative preferred embodiment that simplifies the hardware implementation of the RMS process.
Figure 4 depicts a Read Only Memory (ROM) that requires an input address and provides data output dependant upon the input address. For illustrative purposes only, consider the pixel values to be quantised to 8 bits i.e. 256 individual levels.
The quantised pixels PFO.n and PF1 ,n are used to form an address for the ROM. At each unique address is stored a byte that approximates to the RMS value of PF0,n and PF1 ,n.
It is desirable that the RMS process can be implemented within an ASIC or FPGA. Using the ROM process of Figure 4, and assuming 8 bit RGB video then the number of input-output lines, external to the ASIC or FPGA, required to address the ROM's becomes excessive. Whilst a single ROM could be multiplexed across the R,G and B signals this may cause timing problems. In order to implement the RMS process within an ASIC or FPGA a simplified implementation is disclosed.
An alternative preferred embodiment that simplifies the look up requirement such that the RMS process could be implemented within an ASIC or FPGA may operate as follows:
Given above and below pixels A and B; If A < B then swap A and B such that A is always greater than or equal to
B;
If A = B then the new pixel = A;
Take the difference between A and B;
Use the difference to index into a lookup table; Add the value from the lookup table to B;
Use this result as the value of the new pixel.
In a practical implementation, comprising 8 bit RGB video, the lookup table would be contained in a ROM and the difference information would be used as the address, which would typically be 8 bits, of the data located in the ROM. Thus in this implementation the ROM would contain a maximum of 256 addresses each containing an 8 bit value.
Figures 5 through 8 illustrate this simplified process as follows. In figure 5 the A value is horizontal and the B value vertical. The table contains the difference between A and B where A > B or A = B. Note: In order to simplify the explanation A and B are assumed to take values of between 0 and 100 in steps of 10.
In figure 6 the table depicts the value that would be stored in the lookup
table for each difference between A and B.
Figure 7 shows the effect of applying the previously disclosed method of approximating the RMS value of A and B. In this figure the A value is again horizontal and the B value vertical. The union of A and B within the table is the approximate RMS value of A and B, or a so called "Pseudo RMS" value.
If the results of figure 7 are compared with the true RMS values of A and
B shown in figure 8 ( which has been rounded up to zero decimal places ) it will be seen that the percentage error in most cases is small and not significant in this particular application. This is due to the fact the eye is not sensitive to small variations in colour or intensity.
The objective of this invention is to overcome the previously described shortcomings of simply adding an additional pixel that is the mean of A and B.
This is a achieved by squewing the value of the additional pixel towards the larger value pixel. Since, in the preferred embodiments, the value of the additional pixel is determined by deriving the RMS value of two existing pixels from vertically opposed lines and the RMS value determined, either derived accurately or an approximation, from a lookup table, then the values contained within the lookup table can be altered to provide the most aesthetically pleasing images. In a preferred embodiment, different look up tables, or the same lookup table with different weightings, could be used depending upon the overall characteristics of the original image.
For example, should the overall image be particularly dark then it would be preferable to use interpolated pixels that are closer to the average value. Alternatively, if the image contains areas of high contrast then an alternative table, or the same table with different weightings, may be used with values that enhance the contrast differences.
It will be appreciated by those skilled in the art that these techniques may be applied to the whole image or selectively over the image such that different areas of the image may use different look up tables, or apply different weightings. The use of different tables or weightings could be determined by, but not limited to, brightness, contrast, colour, shading, hue, saturation, or marked
differences between these values over the image being processed.
Whilst the method and apparatus of the present invention has been summarised and explained by an illustrative application in television line doubling, it will be appreciated by those skilled in the art that many widely varying embodiments and applications are within the teaching and scope of the present invention, and that the examples presented herein are by way of illustration only and should not be construed as limiting the scope of this invention.