METHOD AND APPARATUS FOR ELIMINATING COLOUR CROSSTALK
CROSS REFERENCE TO RELATED APPLICATIONS
This application claims the benefit of United States Provisional Patent Application Serial No. 60/174,025 filed December 30, 1999 entitled "METHOD AND
APPARATUS FOR IMPROVING THE QUALITY OF RECONSTRUCTED INFORMATION," of common assignee herewith.
FIELD OF THE INVENTION
The present invention relates generally to electronic image enhancement, and more particularly to processing images obtained from multiple information channels.
BACKGROUND OF THE INVENTION
Photographic film is often used as a medium for recording a source image, and is constructed such that an image is recorded on one or more film layers that each contains photosensitive material, such as solar halide. Color photographic film is constructed using multiple film layers such that different image color information is recorded in different film layers according to the physical composition of each film layer. In general, one layer collects color information on each of the primary colors red, green and blue. Accordingly, these layers are referred to as the red, green and blue layers. Using these different layers effectively divides the source image according to its primary
color components. However, each color layer also records some information from the other two color channels. For example, some red information becomes recorded on the green and blue film layers, some green information becomes recorded on the red and blue film layers, and some blue information becomes recorded on the red and green layers. This information associated with other channels will be referred to as crosstalk, or crosstalk information.
The field of electronic image processing electronically records, or digitizes, a representation of the source image. In conventional color scanning systems, separate red, green and blue sensing elements provide the red, green and blue information in separate information channels. These information channels are subsequently superimposed over each other in order to replicate the image. This process can degrade the resulting image if color separation between the red, green and blue information channels is not adequately maintained. In electronic film development, maintaining color separation between channels is inherently difficult, because of crosstalk, mentioned earlier.
Crosstalk, and particularly color channel crosstalk, can occur during the process of converting an image into a machine readable form. In conventional color scanning systems, the captured image information is typically tagged or otherwise conventionally identified according to its separable color and/or other image attributes, thereby facilitating separate information processing, computer display and printing.
As with electronic film development systems, conventional color scanning systems do not guarantee desired channel separation. Rather, a potential exists in such systems for variances in source medium characteristics such as film or document surface defects, light source or sensor tolerances, light reflection, refraction and absorption variations, and so on. Therefore, as with photographic film, using an image capture system can also result in channel crosstalk.
Noise can also be a problem in converting physical images to electronic representations. While crosstalk causes non-primary information to be included within a given channel, such non-primary information nevertheless is actual image information that is useful in accurately reconstructing the image. In contrast, noise is an image information anomaly, which will detrimentally affect the overall quality of a reproduced image.
Noise combined with crosstalk can be especially problematic, because noise tends tend to spread to all of the channels being operated upon. For example, noise in the red channel affects blue crosstalk information that exists in the red channel. When the blue crosstalk information from the red channel is recombined with the blue information from the blue channel, it brings some varying percentage of the noise from the red channel with it. This transferred noise tends to adversely affect the recombined information.
Accordingly, there remains a need in the art for an improved multiple-channel image processing.
SUMMARY OF THE INVENTION
In accordance with one implementation of the present invention, a method for electronic image processing is provided. In this implementation the information is separately processed for each color channel, on a pixel by pixel basis, such that the information associated with the primary color channel is filtered less than the crosstalk information associated with a non-primary color channel. In a specific implementation, the information associated with the primary color channel is not filtered at all. An advantage of this implementation is that there is generally a minimal loss of image detail, with a significant reduction in crosstalk noise.
In another implementation of this present invention the back, through, and front channels of information obtained from scanning a developing image are processed in order to maximize the source red, green and blue data. Thus, red channel information is obtained by processing back channel information that is filtered less than the through and front channel data. The green channel information is obtained by processing the through channel that is filtered less than the back and front channel data. The blue channel information is obtained by processing front channel information that is filtered less than the back and through channel data. In a preferred embodiment, the primary channel information is not filtered at all.
As a result, at least one embodiment of the present invention decreases crosstalk noise, and the decrease in crosstalk noise that is achieved generally outweighs the loss of image detail that occurs as a result of the non-primary channels being filtered more than the primary color channel.
The present invention also provides a method for improving the quality of reconstructed information. In one embodiment, the method comprises obtaining information from a primary information channel to be reconstructed, and obtaining information from additional information channels. At least a portion of the information from the additional information channels includes information associated with the information obtained from the primary channel. The method further comprises applying a transform to the information obtained from the additional information channels, and combining at least a portion of the transformed information with at least a portion of the primary channel information. Another embodiment of the present invention uses the method just described to generate a digital file tangibly embodied in a computer readable medium.
The present invention further provides a method for correcting image information associated with a plurality of information channels. In a preferred embodiment, the method comprises obtaining image information from a first information channel, and obtaining image information from at least one additional information channel, wherein at least a portion of the information from the at least one additional information channel includes information associated with the information obtained from the first channel. The method further comprises transforming the image information obtained from the at least one additional information channel to obtain transformed image information associated with the first channel, and combining at least a portion of the transformed image information associated with the first information channel with at least a portion of the information from the first information channel to obtain corrected image information associated with the first information channel.
Another embodiment of the present invention implements a digital file tangibly embodied in a computer readable medium. In a preferred embodiment, the digital file is
an improved, reconstructed, or corrected image generated according to a method of the present invention.
Yet another embodiment of the present invention provides for an image processing system comprising at least one illumination source capable of illuminating a physical medium having a plurality of information channels thereon, and at least one detector enabled to generate electrical signals in response to light, the electrical signals representative of image information obtained from a first information channel and at least one additional information channel, wherein at least a portion of the mformation from the at least one additional information channel includes information associated with the information obtained from the first channel, and the detector further enabled to output the electrical signals for image processing. The image processing system also comprises at least one processor, memory operably associated with the processor, and a program of instructions capable of being stored in the memory and executed by the processor. The program of instructions is enabled to transform the image information obtained from the at least one additional information channel to obtain transformed image information associated with the first channel, and to combine at least a portion of the transformed image information associated with the first information channel with at least a portion of the information from the first information channel to obtain corrected image information associated with the first information channel.
An advantage of at least one embodiment of the present invention is that multiple-channel image information can be processed while avoiding a substantial loss of source image information.
Another advantage of the present invention is that the amount of noise added to a primary information channel due to multiple-channel image processing can be reduced.
Yet another advantage of at least one embodiment of the present invention is that color separation between the color channels on color film can be enhanced.
Other objects, advantages, features and characteristics of the present invention, as well as methods, operation and functions of related elements of structure, and the combination of parts and economies of manufacture, will become apparent upon consideration of the following description and claims with reference to the accompanying drawings, all of which form a part of this specification.
BRIEF DESCRIPTION OF THE DRAWINGS
In describing the preferred embodiments of the present invention, reference is made to the following figures, wherein like reference numerals designate corresponding parts in the various figures, and wherein:
FIG. 1 is a functional diagram illustrating a preferred apparatus useful for performing image processing accordance with the present invention;
FIG. 2 is a diagram illustrating a preferred embodiment of an electronic film development scanner according to the present invention;
FIG. 3 is a flowchart illustrating a preferred method for performing image processing according to the invention; and
FIG.4 is a series of three matrices illustrating the combination of three channels according to a preferred embodiment of the present invention.
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT
OF THE INVENTION
In the following detailed description of the preferred embodiments, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that logical, mechanical and electrical changes may be made without departing from the spirit or scope of the invention. To avoid detail not necessary to enable those skilled in the art to practice the invention, the description may omit certain information known to those skilled in the art. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.
The following definitions will aid the reader in properly interpreting the following detailed description of the present invention. The word "light", as used herein, refers to electromagnetic energy, and preferably electromagnetic energy with frequencies generally in the range of 1012 Hz to 1017 Hz, and includes visible light as well as portions of the infrared and ultraviolet spectrum. The term, "transform," is used consistent with its common mathematical meaning, and generally relates to performing systematic alterations of mathematical representations of obj ects according to predetermined rules.
A transform may alter the position, size, or nature of an object by moving it to another location (translation), making it larger or smaller (scaling), turning it (rotation), changing its description from one type of coordinate system to another, and so on. For example, a transform may be used to determine the shape of a sphere projected upon a given plane, or to cross-correlate electronic signals to filter out gaussian noise. "Channel," as used herein, relates to a medium for transferring information. A channel can carry information
(data, sound, video, etc.) in either analog or digital form, and can be a physical link, such as the cable connecting two stations in a network, or it can consist of some electromagnetic transmission of one or more frequencies within a bandwidth in the electromagnetic spectrum, as in radio and television, or in optical, microwave, or voice- grade communication.
FIGS. 1-3 illustrate an apparatus and method for electronic image processing according to the present invention. An electronic image processing system is illustrated in FIG. 1, and designated generally by reference numeral 100. Electronic image processing system 100 may comprise a combination of hardware and software components such as electronic film development scanner 102, and computer 110.
Computer 110 may include I/O devices 116, memory 118, processor 120, output devices 114, and user input devices 112. Computer 110 is preferably capable of executing a program of instructions (commonly referred to as application software) that implements the methods described hereinafter. The application software causes computer 110 to process information, such as image information received from electronic film development scanner 102, conventional scanner 103, or via network 150. In one embodiment, the program of instructions manipulates the image information by applying filtering algorithms to filter out noise from multiple-channel image information before reassembling the information to create the final image. This allows electronic film processing system 100 to produce images with a color and clarity that are otherwise unattainable.
Electronic film development system 100 includes an electronic film development scanner 102, in which front, back and through image information are obtained at different time-scan periods for each negative that is being developed. Referring now to FIG. 2, one embodiment of electronic film development scanner 102 will be discussed. In the embodiment illustrated in FIG. 2, electronic film development scanner 102 comprises table 260, rollers 220, cameras 230, 231, and 232, infrared (IR) lights 240, and extruder
250. All of these elements cooperate to develop film 210 and capture images during the development period.
Table 260 is provided as a support for film 210, and facilitates an even distribution of developer dispensed by extruder 250. In a preferred embodiment extruder 250 extrudes a viscous liquid developing fluid onto film 210. Rollers 220 move film 210 through electronic film development scanner 102. Cameras 230, 231 and 232, and infrared (IR) lights 240 are positioned at appropriate locations along the path of film 210 to capture IR images during various stages of development. Cameras 230, 231 and 232 may be positioned to record images using reflected light, transmitted light, or a combination of reflected and transmitted light provided by IR lights 240. The speed at which various portions of film 210 are moved through image capture hardware 200 may be varied. For example, in FIG. 2, film 210 is moving faster at point A, and slower at point B. The difference in speed is compensated for by allowing film 210 to form a loop at point C. Images captured using electronic film development scanner 102 are provided to computer 110 (FIG. 1) for image processing.
Computer system 110 is shown in FIG. 1 as including user input devices 112, such as a keyboard or mouse, output devices 114 such as a printer or modem, information input/output devices 116, such as a communication interface, for inputting and outputting data, as well as memory 118, such as cache, ram and other computer memory, which can be directly accessible to the processor 120 or alternatively accessible via information input/output devices. The computer system can use a processor such as an Intel Pentium or Pentium II class processor, or other processors, such as those used in other computers, such as RISC and CISC based workstations and the like.
As is well known, the processor 120 operates using an operating system or "OS," such as Microsoft Windows, which operating system then performs system level operations that are desired, typically based upon instructions from an applications
program. One embodiment of the present invention is an application program that implements the functions described hereinafter. It will be appreciated that several variations of the present invention may be carried out within the intended spirit and scope of the present invention. For example, software elements might be implemented in hardware or a combination of hardware and software. Similarly, given performance improvements and other ongoing technological advances, hardware elements might also be replaced by software or a combination of hardware and software.
Overall, in operation, this embodiment of the present invention receives front, back and through image information obtained from the scanning system 102. This information is operated upon by computer system 110, which provides information processing functions, which will be subsequently discussed. In this embodiment, the computer system 110 enables processing of received front, back and through image information to obtain red, green and blue image information that can be combined to create an image for viewing.
As described hereinafter, a preferred embodiment of computer system 110 performs image processing operations sequentially on different pixel data. A pixel is generally recognized as the smallest element that display or print hardware and software can manipulate in creating letters, numbers, or graphics. Those operations necessary to obtain the desired result for a given pixel will be referred to hereinafter as a processing cycle. It should be understood, however, that any number of conventional techniques generally utilized in the computer arts to provide serial and/or parallel processing of information can be similarly applied in accordance with the present invention.
FIG. 3 illustrates operation of the application program on the image information associated with each channel received from the electronic film development system 100.
In step 302 one embodiment of an image processing method according to the present
invention, begins by receiving image information from each of three input channels. The received image mformation is stored in memory 118 such that it can be accessed during subsequent operations. This image information may be input via input/output devices 116 (FIG. 1) and is preferably stored in random access memory (RAM) included in computer system 110. Thereafter, the processor 120 processes each primary image information channel for a given pixel, as described hereinafter, in order to be prepare the channels for subsequent combination.
In electronic film development scanner 102, information for the same pixel of an image may be obtained at different developing times, thereby creating a different set of information at each of the developing times. Ultimately, the information sets obtained at different development times will be combined so that there exists one set of data. According to the present invention, as described hereinafter, the operations that are used to enhance image quality and reduce noise can be implemented on each of the different information sets, the resultant information set, or combinations thereof. The following description encompasses the operations on a single information set. However, it will be appreciated that the described operations may operate repeatedly on the same or different information, and that the steps may be processed concurrently given appropriate computing facilities.
The method proceeds to step 304, where the processor retrieves the channel information associated with the pixel to be operated upon. It should also be understood that while the operation is described as a pixel operation, that a group of pixels could also be used to obtain the pixel values associated with a particular pixel location.
After the method retrieves pixel information in step 304, step 306 is executed to determine which primary channel is to be processed for that pixel. Once this
determination is made, the processor is able to determine proper processing procedures for information pertaining to each of the three color channels.
In particular, in step 308, the processor filters the image information to remove noise from the current non-primary information channels more than the information from the primary color channel. In a preferred embodiment, the information associated with the primary color channel is not filtered at all. For example, if the current primary color channel is the red information channel, then the processor will filter the information from the red channel less than the information from the green and blue channels. In particular, while a bandpass filter will remove frequency components that are outside of the red color frequency range in order to eliminate the undesired (other than red) colors, the non-primary information is also filtered to remove noise that exists in the non-primary channels.
It has been determined that by noise filtering the primary color channel information less than the non-primary color channel data, a significant amount of real image information is maintained, rather than being lost due to noise filtering. Thus, since there is more real image information from the primary color channel, the resultant primary channel pixel value, described hereinafter, will provide for a more accurate representation of the true color associated with that primary channel for that pixel.
Thereafter, in step 310, the information from the primary channel and the non-primary channels is operated upon to obtain a resultant primary channel pixel value for that pixel. This operation is preferably a conventional matrix operation that can use any of a wide variety of known techniques that process crosstalk information in order to obtain an enhanced image. It should be recognized, however, that according to the present invention, although such techniques will also provide a resultant pixel values associated with the respective non-primary channels, such resultant pixel values associated with the non-primary channels are generally not used.
Rather, after a resultant primary channel pixel value for the first primary color channel is obtained, then step 312 determines whether there are any remaining primary color channels that need to be operated upon. If so then step 306 is repeated to determine the new primary color channel, and steps 308 and 310 are performed using a different primary color channel.
Once all resultant primary channel pixel values are obtained for a given pixel, then a decision step 314 determines if any other pixels need to operated upon, such that the above-described operations are repeated for each subsequent pixel. For each successive primary image information channel and each pixel, the computer system 110 retrieves and processes the associated image information and stores the resultant processed data. Following each processing cycle, the processor may delete either the original source image information (i.e. after completion of the processing cycle of the associated data) or the processed source image information obtained from a previous processing cycle (i.e. following completion of a further processing cycle), thereby conserving available space in information storage. Alternatively, such information may be stored using a non-volatile storage device such as a disk or tape drive. Additionally, both processed and unprocessed information may be stored using a computer readable medium, or transmitted to another location via network 150 (FIG. 1).
Referring now to FIG. 4, combining the separate information channels to reconstruct an original image will be discussed. There are seven different matrices illustrated. First input matrix 420, second input matrix 422, third input matrix 424, combination matrix 410, first resultant matrix 430, second resultant matrix 432 and third resultant matrix 434. Also illustrated are reconstructed channel A 450, reconstructed channel B 460, and reconstructed channel C 470.
In accordance with one embodiment of the present invention, first matrix 420 contains three information channels, channel A is the primary information channel that it is desired to reconstruct. Channel BF and CF are non-primary information channels. Channel A is not filtered, so all originally obtained information is retained. Both channel BF and CF have been operated on by combination matrix 410 to produce first resultant matrix 430. First resultant matrix 430 includes reconstructed channels A* , BF* and CF* . Each of the reconstructed channels is a combination of channels A, BF and CF. The two non-primary reconstructed channels BF and CF are not used in this embodiment of the present invention, because a large amount of the information contained in the original channels has been filtered out.
Second matrix 422 and third matrix 424 are similar to first matrix 420, except that each matrix includes a different primary information channel. That is, second matrix 422 includes primary channel B, and third matrix 424 includes primary channel C. In each case, the primary channel is the channel for which reconstruction is desired, and each matrix operation produces a reconstructed information channel corresponding to the primary information channel.
The present invention can also be implemented within a system that also allows for other processing modes to be used. For instance, in a first mode, if noise is not a concern, then conventional processing can be utilized. In a second mode, if the current image processing type is one in which noise is a concern, then the processor can execute a method according to the present invention as described above.
As is apparent from the previous discussion, the invention enables a variety of integrated and stand-alone image processing system implementations. An electronic film development system is a preferred source of image data. However, other information sources are also contemplated as being within the intended scope of the present invention.
For example, the present invention may be implemented utilizing copiers, scanners, optical character recognition systems, facsimile machines, and the like. In a conventional scanning system, the image information can be stored, transferred and/or downloaded, and then subsequently processed using the principles of the present invention as set forth above. Noise that would otherwise be introduced due to channel crosstalk can again be eliminated, without a significant loss of image detail.
While a preferred embodiment of the present invention is described as reconstructing, correcting, or improving a representation of an image originally recorded on color film, it will be appreciated that the present invention is not limited to this embodiment. The present invention may function to reconstruct, improve, transform or correct information from any media employing multiple information channels. For example, modulated broadcast signals make use of multiple sidebands, or channels, and stereophonic recordings use multiple channels to convey information. It should also be noted that the present invention can be used with varying numbers of channels . Although the present disclosure focuses on the use of three color channels, the invention should not be construed to be limited to the use of exactly three channels, and the channels used need not be arranged in layers.
As discussed earlier, at least one method of the present invention uses multiple information channels representative of a physical image, and combines these channels into a transformed, corrected, reconstructed or improved image. This improved image is preferably an electronic representation of the physical image, and is stored as a digital file embodied in a computer readable medium. The improved image contained in the digital file can then be extracted from the computer readable medium and reproduced using a suitable image output device.