WO1994018789A1

WO1994018789A1 - Resolution enhancement system

Info

Publication number: WO1994018789A1
Application number: PCT/US1994/001513
Authority: WO
Inventors: Brian L. Hinman; Jeffrey C. Rodman
Original assignee: Polycom, Inc.
Priority date: 1993-02-11
Filing date: 1994-02-14
Publication date: 1994-08-18

Abstract

A resolution enhancement system (10) comprises a focusing lens (12), a beam-splitter (14), first and second imaging elements (16, 18), and a processing circuit (20). The focusing lens (12) receives and transmits an image to the first and second imaging elements (16, 18). The beam-splitter (14) is interposed between the focusing lens (12) and the imaging elements (16, 18). The beam-splitter (14) divides the image from the focusing lens (12) into two sub-images (24, 26) with each of sub-image being directed to a respective imaging element. The first and second imaging elements (16, 18) are positioned such that each element receives a respective half of the original image from the focusing lens (120). The processing circuit (20) receives the two images which can then be combined at the common edge to form a continuous image with twice the resolution.

Description

RESOLUTION ENHANCEMENT SYSTEM

Background of the Invention

1. Field of the Invention

The present invention relates to a system and method for producing enhanced electronic images. In particular, the present invention is directed to a system using multiple imaging elements to produce an electronic image with greater resolution. The use of multiple imaging elements also provides faster production and processing of the electronic image.

2. Description of the Related Art The processing of an image to produce electronic signals representing the image is well known. Typically, the images are focused on an electronic device such as a charge-coupled device (CCD), vidicon, etc. to produce electronic signals. These signals may then be stored and reproduced to recreate the image. The use of such electronic devices to capture and reproduce images is well known. One continuing problem in electronically storing and reproducing images is providing greater resolution in the quality of the electronic imaged picture. One prior art method used to improve the resolution of electronic images has been to perform further electronic processing on the picture once it has been imaged. For example, the picture may be filtered to increase the apparent sharpness of the picture. A significant drawback to this approach is that resolution is not really enhanced and cannot be greatly improved. The additional filtering only provides improvements in the appearance of the data. If the user wants to focus in on a particular portion of the image, there is not full enhancement of the image. A further problem with this approach is that the additional filtering and processing require additional hardware, and more time to process each image. Such costs are not justified by the nominal enhancement to picture resolution.

Another approach of the prior art creates color pictures by including the use of beam-splitters to produce three identical images, each being an image of the entire scene. Each image is then filtered to pass a particular color primary. Once filtered, each image is passed to a respective electronic imaging device. Often, the filtering and beam splitting requires additional hardware and processing that is not justified by the level of enhancement achieved. Additionally, this method is limited because it is only applicable for the production of color images.

Yet another approach of the prior art attempts to resolve the resolution problem with an imaging device that is moveable to scan the image. The imaging device is mechanically moveable in either one or two dimensions to scan the image. By scanning the image, the number of pixels or points that are electronically stored is increased, thus, there is a true enhancement of the pictures resolution. However, the manufacturing and maintaining of such mechanically movable imaging devices is very difficult and expensive, and the time required to move the apparatus to capture the image is also much longer. An even more basic approach to improve the resolution of an image is increasing the resolution capacity of the electronic imaging device. For example, a common imaging device provides a resolution of 480 pixels by 720 pixels. By using an electronic device with increased resolution (i.e., greater pixel density), there is more data to process and which to zoom in upon. However, the costs of such electronic devices with increased resolution are significantly more expensive than the lower resolution electronic imaging devices.

Therefore, there is a need for an electronic imaging system that can provide true enhancements in picture resolution at low cost.

Summary of the Invention The present invention overcomes the shortcomings and inconveniences of the prior art by providing a system for enhancing the resolution of electronic imaging. The system of the present invention preferably comprises a focusing lens, a beam-splitter, first and second imaging elements, and a processing circuit. The focusing lens is used to receive and transmit an image to the first and second imaging elements. The beam-splitter is interposed between the focusing lens and the imaging elements. The beam¬ splitter divides the image from the focusing lens, with each divided portion of the image being directed to a respective imaging element. The first and second imaging elements are positioned such that each element receives a respective half of the image from the focusing lens. The images from each of the imaging elements can then be combined to form a single image with double the height or width of the image provided by a single imaging element. The processing circuit receives the two images which can then be combined at the common edge to form a continuous image with twice the size.

Many advantages of the present invention will become apparent to those skilled in the art when the following description of the best modes contemplated for practicing the invention is read in conjunction with the accompanying drawings.

-2- Brief Description of the Drawings Figure 1 shows a block diagram of a preferred embodiment for the resolution enhancement system of the present invention;

Figure 2A is a graphical representation of the first image directed from the beam-splitter to the first imaging element;

Figure 2B is a graphical representation of the second image directed from the beam-splitter to the second imaging element;

Figure 2C is a graphical representation of the image produced when the outputs of the first imaging element and the second imaging element are electronically combined;

Figure 2D is a graphical representation of the image projected on the first imaging element;

Figure 2E is a graphical representation of the image projected on the second imaging element; Figure 3 is a block diagram of a second embodiment for the resolution enhancement system of the present invention;

Figure 4 is a graphical representation of the imaged area by each imaging element;

Figure 5 is a graphical representation of the imaged area by each imaging element in a third embodiment;

Figure 6 is a block diagram of a fourth embodiment for the resolution enhancement system of the present invention; and

Figure 7 is a block diagram of a fifth embodiment for the resolution enhancement system of the present invention. Detailed Description of the Preferred Embodiments

Referring now to Figure 1, a preferred embodiment of a resolution enhancement system 10 constructed in accordance with the present invention is shown. The resolution enhancement system 10 preferably comprises a focusing lens 12, a beam-splitter -14, a first imaging element 16, a second imaging element 18, and a processing circuit 20. The system 10 of the present invention advantageously uses a plurality of imaging elements 16, 18 and the beam-splitter 14 to increase the resolution of picture by a factor of two. The beam-splitter 14 divides the image into halves, with half of the image directed to the first imaging element 16 and the other half of the image directed to the second imaging element 18. Thus, twice the amount of pixel data is gathered and stored for each image to greatly improve the resolution of the picture.

The image to be processed is preferably received through the focusing lens 12. The focusing lens 12 is optically coupled to transmit the image to the

-3- beam-splitter 14. The focusing lens 12 is preferably a retrofocus or reverse telephoto lens to provide a more compact system. The focusing lens 12 is used to focus the image through the beam-splitter 14. The beam-splitter 14 in turn divides the image such that half the image is then transmitted to each of the imaging elements 16, 18. The beam-splitter 14 is preferably a half-silvered prism. The beam-splitter 14 is optically coupled to both imaging elements 16, 18 to transmit a portion of the image to each.

Once the image has been divided, the halves that comprise the image are transmitted to the first and second imaging elements 16, 18, respectively. While the two images from the beam-splitter 14 are referred to as halves, it should be understood that the images may be various areas of the original image and are not necessarily 50% each. The imaging elements 16, 18 are preferably electronic elements that convert light waves into electrical signals such as charge-coupled devices (CCD), vidicons, or comparable imaging tubes. In an exemplary embodiment, each imaging element 16, 18 is a CCD with resolution of 480 by 720 pixels. In the system 10 of the present invention, the first and second imaging elements 16, 18 are positioned with respect to the beam-splitter 14 such that the images formed on the first and second imaging elements 16, 18, when electrically combined at their common edge, form a continuous image of with twice the original width(or height). For example, in the system shown in Figure 1, the first imaging element 16 is positioned to receive the left half of the image from the beam-splitter 14 while the second imaging element 18 is positioned to receive the right half of the image from the beam-splitter 14. Thus, the system of the present invention doubles the resolution of the image using two low resolution imaging elements. This is particularly advantageous because low resolution imaging elements are significantly less costly than imaging elements with double the resolution of a low resolution imaging element.

In the preferred system 10 of Figure 1, the imaging elements 16, 18 are precisely positioned with respect to the two images projected by the beam¬ splitter 14 to assure that: 1) each imaging element 16, 18 receives a different portion of the image, and 2) that the imaging elements are precisely aligned to receive a common edge 22 of the image (or with an overlap proximate the common edge) so that the outputs of the imaging elements 16, 18 can be combined to create one continuous image. In the exemplary embodiment, the first imaging element 16 is positioned on a first side of the beam-splitter 14, and the second imaging element 18 is positioned on a second side of the beam¬ splitter 14. As shown in Figure 1, the first and second sides of the beam-splitter

-4-

SUBSTITUTE SHEET (RULE 14 are preferably perpendicular to each other. Additionally, the first imaging element 16 has an edge 24 that is aligned with the common edge 22 of one image from the beam-splitter 14, and the second imaging element 18 has an edge 26 aligned with the common edge 22 of another image from the beam- splitter 14.

Referring now to Figures 2A-2E, the images received and processed by the first and second imaging elements 16, 18 will be discussed. It should be understood that the beam-splitter 14 produces two images of the picture received from the focusing lens 12. For example, if an image of a polygon, as in Figures 2A-2E, were transmitted by the focusing lens 12, the beam-splitter 14 would divide the image to produce two images. Figure 2A shows a graphical representation of the image of the polygon projected toward the first imaging element 16. The polygon is used only as an example, and the image may be any scene the user wants to electronically image. The beam-splitter 14 also projects the mirror image of the polygon, as shown in Figure 2B, toward the second imaging element 18. Each imaging element 16, 18 processes a respective half of the image on either side of the common edge 22 (delineated by the dashed lines in Figures 2A and 2B). The image projected onto and processed by the first imaging element 16 is shown in Figure 2D. The image projected onto and processed by the second imaging element 18 is shown in Figure 2E. The images produced by each imaging element 16, 18 are then processed by the processing circuit 20, and rejoined to produce the image shown in Figure 2C. As clearly shown, the amount of data that can be used to zoom in on a portion of the picture has been doubled by having each imaging element 16, 18 process half the picture received from the focusing lens 12.

The processing circuit 20 provides a plurality of inputs with at least one input coupled to each imaging element 16, 18. The transformation of the image from light to electrical signals is performed by the imaging elements 16, 18, which output the signals to the processing circuit 20. The processing circuit 20 also performs filtering and other processing on the signals from the imaging elements 16, 18. Further processing of the output of the imaging elements 16, 18 is required because different imaging elements will exhibit different brightness and contrast characteristics. These characteristics can vary from line to line and from column to column. The processing circuit 20 preferably includes circuits to compensate for these variations to produce a perfect or "seamless" match between the transformation of the images performed by each imaging element 16, 18. For example, the processing circuit 20 may include memory and other circuitry for storing correction values in a

-5- look up table, and reading the values out either as the image is read out, or applying the correction values later if the image is to be stored. The processing circuit 20 preferably includes two independent processing subsystems for such processing. Each processing subsystem is dedicated to a respective imaging element 16, 18. Thus, each half of the image may be processed simultaneously. If the image is to be used in a digital system, this provides a doubling of the bandwidth for processing the image without having to increase the clock rate.

As note above and shown in Figure 2E, the image received by the second imaging element 18 is reversed with respect to the image seen by the first imaging element 16. The processing circuit 20 also includes circuitry to correct this reversal. For example, the scan direction for the second imaging element 18 may be reversed, or the image may be digitized and read back out in reverse pixel order. Those skilled in art will realize that the system may alternately include a relay lens or a lens-and-mirror system to optically reverse the image before it is received by the second imaging element 18. This reversal can be accomplished after digitizing by reading each scan line from the digitizer memory in reverse order. The processing circuit 20 is a conventional circuit used with CCD's as know in the art. The processing circuit may include memory and a digitizer as known to those skilled in the art. Referring now to Figure 3, a block diagram for a second embodiment of the resolution enhancement system 30 is shown. The second embodiment of the system 30 preferably comprises a focusing lens 32, a plurality of beam¬ splitters 34, 36, 38, a plurality of imaging elements 40, 42, 44, 46, and processing circuit 48. The second embodiment of the system 30 of the present invention advantageously uses a plurality of beam-splitters 34, 36, 38 to further divide the original image received from the focusing lens 32 onto four imaging elements 40, 42, 44, 48. This increases the resolution of the system 32 by a factor of four since each of the imaging elements 40, 42, 44, 48 transforms a quarter of original image. In the second embodiment of the system 32, the original image is received and transmitted by the focusing lens 32 onto the first beam-splitter 34. The focusing lens 32 is optically coupled to transmit the original image onto the first beam-splitter 34, and the first beam-splitter 34 divides the original image into two images about equal in size. The first beam-splitter 34 is optically coupled to the second and third beam-splitters 36, 38 to project the left half and the right half of the original image onto the second and third beam¬ splitters 36, 38, respectively. The second beam-splitter 36 is also optically coupled to the first and second imaging elements 40, 42, and again divides the left half of the original image received from the first beam-splitter 34 to project half of the left half of the image (i.e., a quarter of the original image) onto each of the first and second imaging elements 40, 42. Similarly, the third beam¬ splitter 38 is optically coupled to the third and fourth imaging elements 44, 46, and divides the right half of the original image received from the first beam¬ splitter 34 to project a quarter of the original image onto each of the third and fourth imaging elements 44, 46. The outputs of the four imaging elements 40, 42, 44, 46 are coupled and output to the processing circuit 48 for producing an electrical signal with four times the resolution of a single imaging element. As shown in Figure 4, the original image is divided along two common edges 50, 52 that are perpendicular to each other. Each of the four areas is advantageously processed by a respective imaging element 40, 42, 44, 46 for the increased resolution. Similar to the preferred embodiment, some of the images processed by the imaging elements 40, 42, 44, 46 will be reversed as compared to the original image received by the first beam-splitter 34. Since the original image had been divided multiple times the image may be reversed about the X and Y axis of the image(the common edges 50, 52 of the image). The processing circuit 48 also includes circuitry to correct this reversal problem. The scan direction for the second and third imaging elements 42, 44 may be reversed, or the image may be digitized and read back out inverse pixel order to correct the problem. Finally, those skilled in the art will realize that additional lenses and mirrors may be interposed between the beam-splitters 34, 36, 38 to correct the reversal problem, and also correctly focus the output of the first beam-splitter 34 onto the second and third beam-splitters 36, 38. A third embodiment of the system is suggested by Figure 5. The third embodiment of the system is substantially similar to the preferred embodiment shown in Figure 1. However, in the third embodiment, each imaging element 16, 18 processes slightly more than half of the original image to provide a zone of overlap 64. For example, the first imaging element 16 images an area from the right edge of the image in Figure 5 to a first edge 60. The second imaging element 18 processes an image from the left edge of the image in Figure 5 to a second edge 62. The area between the first and second edges 60, 62 defines the zone of overlap 64. The signals gathered by the first and second imaging elements 16, 18 for corresponding areas (the same optical element) in the zone of overlap 64 are then compared by the processing circuit 20 to determine the amount of correction to be applied to the signals produced by the first and second imaging elements 16, 18. For example, the brightness and contrast corrections for each line and column can easily be calculated from the zone of overlap 64, thereby assuring that the combination of the outputs from the first and second imaging elements 16, 18 will be completely seamless.

The zone of overlap 64 can also be used to analyze and compensate for geometric mismatches between the two images, such as would be observed if the alignment between the first and second imaging elements 16, 18 is slightly flawed. There are various ways of achieving this as known to those skilled in the art. One method is to cross-correlate the common area between the two images, and calculate the correction in terms of X, Y, and Z offsets and rotations as required for a perfect match, then apply those transformations to the subject half of the image.

Having described the preferred embodiments of the present invention, it will be understood to those skilled in the art that many modifications and variations thereto are possible, all of which fall within the true spirit and scope of this invention. The above description shall not be construed as limiting the ways in which this invention may be practiced but shall be inclusive of many other variations that do not depart from the broad interests and intent of the invention. For example, as shown in Figure 6, a fourth embodiment of the present invention may employ a lens, three CCDs, and three corresponding beam splitters to divide the image into three equal segments. The output of the three CCDs can then be coupled by the processing circuitry into a single image with three times the pixel density of a normal low resolution imaging device. Similarly, Figure 7 shows a fifth embodiment of the present invention. In the fifth embodiment the beam-splitter of the preferred embodiment is complemented by a mirror 70 and a series of lenses 72 to focus the image on each imaging device. The mirror 70 and lenses can eliminate the need for reversal and other geometric compensation circuitry.

-8-

Claims

WHAT IS CLAIMED IS:

1. A resolution enhancement system for increasing the resolution of an electronically imaged picture, said system comprising: a lens for receiving, transmitting, and focusing an optical image; a beam-splitter for dividing the optical image into a first portion and a second portion, said beam-splitter optically coupled to the lens to receive the optical image, said first and second portions comprising two separate but adjacent geometric regions of the image; a first imaging element having an output for converting an image into electrical signals, said first imaging element optically coupled to the beam¬ splitter to receive the first portion of the optical image; a second imaging element having an output for converting an image into electrical signals, said second imaging element optically coupled to the beam-splitter to receive the second portion of the optical image; and a circuit having inputs and an output for processing and combining the signals from the first and second imaging elements to produce electrical signals representing the optical image from the lens, one input of the circuit coupled to the output of the first imaging element, and another input of the circuit coupled to the output of the second imaging .

2. The system of claim 1, wherein the first and second imaging elements are positioned with respect to the beam-splitter such that the images are converted to electrical signals, when combined at their common edge form a continuous image of increased size.

3. The apparatus of claim 2, wherein the first imaging element is positioned in a plane perpendicular to the plane in which the second imaging element is coupled.

4. The apparatus of claim 3, wherein the first and second imaging elements are charged-coupled devices.

5. The apparatus of claim 3, wherein the first and second imaging elements are vacuum-tube type imaging devices.

6. The apparatus of claim 1, wherein the circuit filters and corrects the signals output by the first and second imaging elements to compensate for different characteristics in brightness and contrast between the first and second imaging elements for a seamless match between the images produced by the first and second imaging elements.

7. The apparatus of claim 1, wherein the circuit includes memory for storing correction values, and circuitry for applying correction values to the signals output by the first and second imaging elements.

8. The apparatus of claim 2, wherein the circuit applies correction values to the signals output by the first and second imaging elements to compensate for geometric mismatches in alignment between the two images.

9. The apparatus of claim 2, wherein the circuit comprises: a first independent processing subsystem having an input and an output for processing the signals output by the first imaging element, the input of the first independent processing subsystem coupled to the output of the first imaging element; and a second independent processing subsystem having an input and an output for processing the signals output by the second imaging element, the input of the second independent processing subsystem coupled to the output of the first imaging element.

10. The apparatus of claim 1, wherein: the first portion of the optical image processed by the first imaging element includes an overlap area; wherein the second portion of the optical image processed by the second imaging element includes the overlap area that is imaged by the first imaging element; and wherein the circuit uses the outputs of the first and second imaging elements for the overlap area to calculate and apply correction values to the outputs of the first and second imaging element to compensate for differences in contrast and brightness characteristics of the first and second imaging elements.

11. A resolution enhancement system for increasing the resolution of an electronically imaged picture, said system comprising: a lens for receiving, transmitting, and focusing an optical image; a first beam-splitter for dividing the optical image into a first portion and a second portion, said beam-splitter optically coupled to the lens to receive the optical image; a second beam-splitter for dividing the first portion of the optical image into a third portion and a fourth portion, said second beam-splitter optically coupled to the first beam-splitter to receive the first portion of the optical image; a third beam-splitter for dividing the second portion of the optical image into a fifth portion and a sixth portion, said third beam-splitter optically coupled to the first beam-splitter to receive the second portion of the optical image;

-0J3- . a first imaging element having an output for converting an image into electrical signals, said first imaging element optically coupled to the second beam-splitter to receive the third portion of the optical image; a second imaging element having an output for converting an image into electrical signals, said second imaging element optically coupled to the second beam-splitter to receive the fourth portion of the optical image; a third imaging element having an output for converting an image into electrical signals, said third imaging element optically coupled to the third beam-splitter to receive the fifth portion of the optical image; a fourth imaging element having an output for converting an image into electrical signals, said fourth imaging element optically coupled to the third beam-splitter to receive the sixth portion of the optical image; and a circuit having inputs and an output for processing and combining the signals from the first, second, third, and fourth imaging elements to produce electrical signals representing the optical image from the lens, one input of the circuit coupled to the output of the first imaging element, one input of the circuit coupled to the output of the second imaging element, one input of the circuit coupled to the output of the third imaging element, and one input of the circuit coupled to the output of the fourth imaging element.