WO1998015126A1

WO1998015126A1 - Reduced cost high resolution color camera

Info

Publication number: WO1998015126A1
Application number: PCT/US1997/017670
Authority: WO
Inventors: Karl L. Chase
Original assignee: Honeywell Inc.
Priority date: 1996-09-30
Filing date: 1997-09-30
Publication date: 1998-04-09
Also published as: CA2264777A1; JP2001506812A; EP0929977A1

Abstract

A high resolution color camera resulting from the combination of a high resolution monochrome camera output with a less high resolution lower cost color camera output by exposing the two cameras to the same scene and modulating the intensities of the outputs from the monochrome camera pixels with the color component outputs from the color camera pixels over approximately the same pixel areas. The combiner compares the output from the color camera with the output from the monochrome camera and adjusts at least one of the cameras to assure that the scenes viewed by both cameras are substantially the same.

Description

REDUCED COST HIGH RESOLUTION COLOR CAMERA BACKGROUND OF INVENTION

The Government has rights in this invention pursuant to Contract No. NAS 1-20219, awarded by NASA. 1. Field of the Invention

The present invention relates to cameras and particularly color video cameras which have high resolution and are yet reasonably inexpensive. 2. Description of the Prior Art

Some still cameras, and most video cameras, operate by focusing an image of a remote scene on a plurality of small light detectors such as CCDs each of which operates to produce an output indicative of the amount of light it receives. In monochrome cameras, there is one intensity for each detector while in color cameras an intensity for each of the three primary colors, red, green and blue is produced. The output from each detector constitutes a pixel which may be presented to a viewing device such as a TV monitor. The resolution seen by the viewer is dependent on size and the number of pixels being viewed. Color cameras such as conventional TV cameras produce about 480 lines each having 640 pixels which is not considered very high resolution. For purposes of producing a high fidelity image, perhaps five times as much resolution is needed, e.g. 2000 lines with 3000 pixels each. Monochrome cameras with this resolution are fairly easy to construct and are not very expensive but color cameras capable of high resolution are quite complex, rather bulky and very expensive.

BRIEF DESCRIPTION OF THE INVENTION

The present invention combines a high resolution monochrome camera with a lower resolution color camera to produce a combined image which is discerned by the viewer to be high resolution color. Both cameras are trained on the same object and the relatively inexpensive monochrome camera producing a high number of pixels produces one intensity signal for each detector. The relatively inexpensive relatively low resolution color camera produces far fewer pixels and has three outputs per pixel, one for red, one for green and one for blue. The single intensity signal from the monochrome camera is divided into three signals each of which is modified by the intensity of one of the color signals. The resultant output contains three signals for each of the higher number of pixels which is presented to the viewer. It is well known that the human eye needs less color resolution than intensity resolution and accordingly, while each pixel in the resultant image may not be exactly the color of the object at that point, it will be imperceptibly different to the eye.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 shows a block diagram of the present invention; Figure 2 shows one possible pixel arrangement for the two cameras of Figure 1; and Figures 3, 4 and 5 show the color intensities, the monochrome intensities and the combined intensities of two of the pixels of Figure 2 respectively.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In Figure 1, a color camera 12 and a monochrome camera 14 are both shown with their fields of view trained on an area A of a remote scene 16. Color camera 12 may be a relatively low cost camera such as a conventional TV camera with relatively low resolution of say 512 by 512 pixel image. Generally, the pixel image is produced by an array of detectors, such as CCD's each having a small light receiving area (1 pixel) and each producing an output which is used to energize a small area (1 pixel) on a viewer such as a CRT. In the case of color camera 12, the individual pixel signals will normally contain three components, Red, Blue and Green in order to provide color signals for the viewer. Monochrome camera 14 may be a relatively low cost camera with a relatively high resolution of say 2048 by 2048 pixel image. In the case of monochrome camera 14, the individual pixel signals will normally only be indicative of the intensity of light falling on the detector. These numbers, 512 and 2048, have been chosen for convenience in the explanation to follow and the actual number of pixels that each camera produces may vary with the desired resolution and the cost/availability of the cameras.

The outputs from color camera 12 and monochromatic camera 14 are produced on lines 20 and 22 respectively and are presented to a combiner 26. Combiner 26 may operate either on the analog signals from the cameras 12 and 14 or may include a digitizer which operates to produce digital signals indicative of the analog magnitudes of the signals from the individual pixels in the cameras 12 and 14. Combiner 26 may also include a micro processor which processes the signals from the cameras 12 and 14 in various ways. For example, combiner 26 may compare the wave forms from the two cameras to determine a "best fit" situation and to adjust one or both of the cameras as by a connection shown as dashed line 28, to assure that they are viewing the same scene as closely as possible. The combiner 26 also operates to modify the intensity signals from the monochrome camera 14 in accordance with the three color components for each pixel signal from the color camera. For example, as seen in Figure 2, a small portion 30 of the overall pixel image from the color camera 12 is shown containing 12 pixels arranged in a rectangular form. A substantially equal sized pixel image from the monochrome camera 14 is shown containing 48 pixels in rectangular form and represents the same area of the remote scene as is seen by the color camera 12. Although areas 30 and 36 are substantially the same size, portion 36 for monochrome camera 14 contains four times the number of pixels as the color portion 30 for color camera 12. This gives much greater resolution to camera 14. Each of the pixels of the color portion 30 produces a signal having three components, representing the intensity of red, green and blue received by each detector pixel and these combine to give each color pixel its desired color. Each of the pixels from the monochrome portion 36 has a single component indicative of the intensity of light on that pixel. As seen in Figure 2, an area 40 from portion 30 is located in the lower left corner of the portion 30 and area

46 is similarly located in the portion 36. A line 50 is shown representing that the R,G and B values from area 40 are to be combined with the four intensity values from the pixels in area 46 as will be described in connection with Figures 3, 4 and 5.

While ideally, the size of the area 40 and 46 are the same, in the real world this does not normally happen exactly. Camera distortions such as keystone and pincushion effects make the overlay correspondence only approximate. In order to combine the two images, it may be necessary to reduce the misalignment. This may be done by first obtaining the X-Y coordinates of the monochrome pixels, Xm and Ym and the color pixels Xc and Yc. Pick the coordinates for the color pixels that most closely overlay the monochrome pixels. This can be done by calculating Xc = a(Xm) + b(Ym) +c and Yc = d(Xm) + e(Ym) +f where a,b,c,d,e and f are constants determined from calibration. Alternately, a random lookup table associating the Xm and Ym positions with the Xc and Yc positions may be used. Next, the color information should be normalized so that luminance of the sum of red, blue and green components is a constant value and then calculate the ratio each color component has to the whole and use the ratio as a multiplier to determine a normalized red, blue and green value which may then be multiplied by the individual pixel values of the monochrome array. These functions can be easily programmed into the microprocessor in combiner 26.

In Figure 3, area 40, from Figure 2, is shown having an output consisting of three components: the red component, R, shown by arrow 54 is relatively large, the green component, G, shown by arrow 56 is relatively small and the blue component, B, shown by arrow 58 is in-between. These components combine to produce a particular color to be displayed.

In Figure 4, the area 46, from Figure 2, is shown having four pixels 60, 62, 64 and 66 each of which is 1/4 the size of the pixel 40 in Figure 3. Pixel 60 produces a single output, 1, shown by arrow 70 indicative of the intensity of light for pixel 60, pixel 62 produces a single output, 2, shown by arrow 72 indicative of the intensity of light for pixel 62, pixel 64 produces a single output, 3, shown by arrow 74 indicative of the intensity of light for pixel 64 and pixel 66 produces a single output, 4, shown by arrow 76 indicative of the intensity of light for pixel 66. It is seen that pixel 60 is relatively dim, pixels 62 and 66 are somewhat brighter, while pixel 64 is brightest. Figure 5 shows an example of what happens when the combiner operates on the values from the monochrome and color intensities as described above. More particularly, the intensities of light 70, 72, 74 and 76 from Figure 4 are combined, as described above, with the color components 54, 56 and 58 of Figure 3. In Figure 5, and area 80, which is the same size as areas 40 and 46, is shown divided into four pixels 82, 84, 86 and 88. Pixel 82 is shown producing three components each of which is a combination of the R, G and B components of Figure 3 and the intensity 70 of Figure 4. It is seen that a relatively large red component, Rl, shown by arrow 101, a relatively small green component, Gl, shown by arrow 103 and a middle sized blue component, Bl, shown by arrow 105 represent the light for pixel 82. These components will combine to produce a color close, but probably not exactly the same as the color from pixel 40 of Figure 3. Pixel 84 is shown producing three components each of which is a combination of the R, G and B components of Figure 3 and the intensity 72 of Figure 4. It is seen that a relatively large red component, R2, shown by arrow 111, a relatively small green component, G2, shown by arrow 113 and a middle sized blue component, B2, shown by arrow 115 represent the light for pixel 84. These components will also combine to produce a color close to, but probably not exactly the same as, the color from pixel 40 of Figure 3. Pixel 84 is shown producing three components each of which is a combination of the R, G and B components of Figure 3 and the intensity 74 of Figure 4. It is seen that a relatively large red component, R3, shown by arrow 121, a relatively small green component, G3, shown by arrow 123 and a middle sized blue component, B3, shown by arrow 125 represent the light for pixel 86. These components will also combine to produce a color close to, but probably not exactly the same as, the color from pixel 40 of Figure 3. Finally, pixel 88 is shown producing three components each of which is a combination of the R, G and B components of Figure 3 and the intensity 76 of Figure 4. It is seen that a relatively large red component, R4, shown by arrow 131, a relatively small green component, G4, shown by arrow 133 and a middle sized blue component, B4, shown by arrow 135 represent the light for pixel 88. These components will also combine to produce a color close to, but probably not exactly the same as, the color from pixel 40 of Figure 3.

As an example, assume that the red component 54 is 6 units in length, the blue component is 4 units in length and the green components 2 units in length. Assume also that the output 70 of Figure 4 is 10 units, output 72 is 16 units, output 74 is 20 units and output 76 is 18 units. Combiner 26 operates to produce the ratios 6/12 for red, 4/12 for blue and 2/12 for green which are then used to adjust the monochrome intensities 70, 72 , 74 and 76 accordingly. Accordingly, intensity 101 in Figure 5 would be proportional to 6/12(10), output 103 would be proportional to 2/12(10) and output 105 would be proportional to 4/12(10). Similarly, output 111 would be proportional to 6/12(16), output

113 would be proportional to 2/12(16) and output 115 would be proportional to 4/12(18). This would apply to the outputs of pixels 64 and 66 also with the result that the colors of each of the resulting pixels 82-88 would be slightly different from each other and all about the same as pixel 40. It should be noted that a color may have a value of zero in Figure 3, in which case, the computer would be programmed to recognize this condition and to make the resulting component zero rather than trying to divide by zero. Other schemes may be used to combine the color values from the color camera 12 with the monochrome values from the camera 14. For example, if somewhat less accuracy is permitted, merely adding the values of Figure 3 to the values of Figure 4 might be considered.

Thus it is seen that the combiner 26 of Figure 1 produces the outputs 101 - 135 on a line 140 to a utilization device such as a viewer 144 which may be a CRT with the higher resolution produced by the monochrome camera 14 with the color from the color camera 12. In other words, there will be four pixels in the output signal on line 144 for each of the color pixels such as 40 from Figure 2 and the color of these pixels will be substantially the same as the color of pixel 40. The human eye will not be able to discern the different colors even though they will vary slightly from pixel to pixel. After all pixels in the image have been processed and submitted to the viewer 144, the resultant image will appear of high resolution and excellent color without the expense of a high resolution color camera.

As mentioned above, the two cameras 12 and 14 can be automatically aligned by comparing the peaks and the valleys in their output signals and moving one or both cameras until a "best fit" occurs. Adjustments can also be made electronically by altering the positions of the signals in the combiner. The quality of the image can also be monitored by comparing the two independent signals from the camera. The present invention also assures a "fail-safe" system in the event that either camera fails, because the other camera will still provide an emergency picture which will be temporarily satisfactory even though it may lack either color or high resolution.

These and many other advantages will occur to those skilled in the art and many changes may be made without departing from the spirit of the present invention. For example, while a ratio of 4 monochromatic pixels to one color pixel has been chosen to explain the invention, other ratios may also be used. Furthermore, to use a color pixel which is four times as large as the monochromatic pixel may not always be possible. As explained above, slight mismatches in the optics of the cameras may cause the color pixels not to linearly correspond with the monochrome pixels and may correspond to an approximate but not necessarily rectangular group of monochrome pixels. The resulting color to the individual output pixels will still be quite close to the desired color and indistinguishable by the average human eye.

Accordingly, I do not wish to be limited to the disclosures used in connection with describing the preferred embodiment.

Claims

The embodiments of the invention in which an exclusive property or right is claimed are defined as follows: 1. An improved camera system comprising: a color camera having a first plurality of pixels and positioned to view a first area in a scene, the output of each pixel containing 1st, 2nd and 3rd color components of magnitude indicative of the intensitity of light of each color thereon; a monochromatic camera having a second plurality of pixels greater than the first plurality and positioned to view the first area, the output of each of the second plurality of pixels having a magnitude indicative of the intensity of light thereon; a combiner connected to receive the outputs of the color and nonochromatic cameras and to combine them so that each output of a pixel of the monochromatic camera is modified by each of the color components of the color camera to produce a combined signal having three color components for each pixel of the second plurality.

2. The camera system of claim 1 further including utilization means to receive the combined signals and provide a high resolution picture therefrom.

3. The camera system of claim 1 wherein the combiner compares the output from the color camera with the output from the monochrome camera and adjusts at least one of the cameras to assure that the scenes viewed by both cameras are substantially the same.

4. The camera system of claim 1 wherein the combined signals are produced by determining the ratios of the 1st, 2nd and 3rd color components with the sum of the 1st, 2nd and 3rd color components and multiplying the output from the correspondingly positioned pixels from the monochrome camera by the ratios.

5. The camera system of claim 1 wherein the first plurality of detector pixels is substantially equal to at least four times the second plurality of detector pixels.

6. The method of providing a low cost high resolution camera system with a low cost low resolution color camera and a low cost high resolution monochrome camera comprising the steps of:

A. positioning the two cameras to observe the same scene and produce a color output signal and a monochrome output signal respectively;

B. modifying the monochrome output signal with the color output signal to produce a resultant color signal; and

C. producing a picture of the scene from the resultant signal.

7. The method of claim 6 wherein step B. includes the step of:

B2. determining the color components of each pixel in the color output signal and the intensity components of each pixel in the monochrome output signal; and

B3. multiplying the intensity components by the color components to produce color components for each pixel in the resultant color signal.

8. The method of claim 7 wherein step B2. includes the step of:

B2a. obtaining the ration of each color component to the sum of the color components of each pixel in the color output, and step B3. include the step of :

B3a. multiplying the intensity components by the rations obtained in step B2a.

9. The method of claim 6 further including the step of :

D. Comparing the color output signal and the monochrome output signal and adjusting the position of at least one of the cameras in Step A. so as to obtain a "best fit".