JPH1176159A - Image forming device comprising combination of plural imaging element - Google Patents

Abstract

PROBLEM TO BE SOLVED: To form an image of high resolution from double the number of pixels of a conventional imaging element by using a conventional imaging element. SOLUTION: An image forming device is provided with a half mirror 11 separating the optical path of an objective optical system 10 into two; a first CCD 12 and a second CCD 13 which are arranged within the two separate optical paths of the half mirror 11 to form image signals consisting of data about different pixel positions; and an image combining circuit 23 forming an image with a large number of pixels by combining the image signals obtained with the CCD 12, 13. A defective pixel forming circuit by which defective pixels other than the pixels set for the CCD 12, 13 are computed and generated from peripheral pixel signals is provided in the image combining circuit 23. This constitution enables the formation of an image with, e.g. four times the number of pixels conventionally set for the CCD 12, 13.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image forming apparatus in which a plurality of image pickup devices are combined, and more particularly, to an image forming apparatus capable of obtaining a high-resolution image with a multiplied number of pixels in an electronic endoscope or the like.

[0002]

2. Description of the Related Art A CCD (Charge Coupled Device) or the like is used as an image pickup device of an electronic endoscope, a video camera or the like, and the resolution of a subject image depends on the number of pixels of the CCD. In recent years, CCDs having a size of about 1/2 inch and having 1.3 million pixels have been manufactured. CCDs having a large number of these pixels are used for photographing high-definition images.

[0003]

By the way, the above-mentioned CCD having a large number of pixels requires a special driving circuit and a signal processing circuit, and is not applicable to an electronic endoscope whose diameter is limited. At present, it is difficult, and the cost is high.

On the other hand, if an in-vivo image captured by an endoscope is displayed on a high-vision screen, a high-quality and easy-to-view image can be obtained, which is also useful as an observation image for diagnosis and treatment. . Therefore, conventionally used,
For example, if a high-resolution image corresponding to a larger number of pixels can be obtained using a CCD having about 400,000 pixels, a device with reduced cost can be obtained without the need for a special circuit.

SUMMARY OF THE INVENTION The present invention has been made in view of the above points, and has as its object to utilize a conventionally used image pickup device and to provide a high-resolution image with the number of pixels doubled as compared with the number of pixels of this image pickup device. An object of the present invention is to provide an image forming apparatus in which a plurality of image pickup devices capable of forming an image are combined.

[0006]

According to a first aspect of the present invention, there is provided an image forming apparatus in which a plurality of image pickup devices are combined to separate an optical path on an image receiving side from one objective optical system. (For example, a half mirror) and a plurality of imaging devices arranged on the optical paths separated by the optical path separating device and arranged to form image signals composed of data at different pixel positions in the subject image. And an image synthesizing circuit for synthesizing image signals obtained from the plurality of image sensors and forming an image having a larger number of pixels than the number of pixels of the image sensors. The invention according to a second aspect is characterized in that the image synthesizing circuit calculates and generates a missing pixel other than the pixel set by the image sensor from surrounding pixel signals.

According to the above arrangement, for example, two CCDs are attached to one objective optical system via a half mirror, and these CCDs are arranged at different positions so that a pixel unit forms a checkered pattern. You. In other words, in a state where the checkerboard square is equivalent to a quarter of the pixel set by the CCD, the squares are obliquely shifted from each other, and the checkerboard has the same color (for example, a black and white checkerboard black). Are arranged so that is formed. According to this, the size of one pixel is 1/4 of one pixel set by the CCD, and two CCDs can obtain, for example, pixel information of a black portion in a pine pattern. Therefore, in this case,
An image signal having twice the number of pixels of the CCD is formed.

The white part of the checkered pattern is a CCD.
In this case, a missing pixel from which information cannot be obtained can be obtained. For this missing pixel, a pixel signal can be generated from surrounding pixel signals, for example, four pixels vertically and horizontally or two pixels horizontally. According to this, an image signal having four times the number of pixels of the CCD is formed.

[0009]

FIG. 1 shows a circuit configuration of an image forming apparatus in which a plurality of image pickup devices according to an embodiment are combined, and FIG.
1 shows a detailed configuration of the image synthesizing circuit in FIG. 1, and this device is applied to an electronic endoscope. In FIG. 1, an objective optical system 10 is disposed at the end of the endoscope, and a half mirror (optical path separation element) 11 that separates an image receiving side optical path into a transmitted optical path and a reflected optical path is provided behind the objective optical system 10. Is arranged. Instead of the half mirror 11, another optical path separating element such as a beam splitter can be used. The half mirror 11 has a first CCD 12 on its transmitted light path side and a second CCD 1 on its reflected light path side.
3 are optically connected, and these CCDs 12, 1
For example, a device having 400,000 pixels is attached as the third device.
These CCDs 12 and 13 are one CCD driver 1
4 is similarly driven and controlled.

FIG. 3 shows the first and second CCDs 12,
13 shows the mounting relationship with respect to the image light path of the image pickup device 13. In this figure, the CCDs 12 and 13 are placed on the optical path of the objective optical system 10 of the optical axis 100. In this example, l1 × l2 originally set by the CCDs 12 and 13 are used.
In order to set pixels a and b (a = b) each having a quarter of the size of the pixel A having the size of the pixel A by the size of the pixel a (= b), The CCDs 12 and 13 are attached and arranged so as to be obliquely shifted from each other. According to this, as shown in the lower right of FIG. 3, for example, the pixels a and b form a black-and-white checkerboard pattern of the same color (black or white). Note that this C
For the CDs 12 and 13, there is no need to change the configuration of the internal photodiode and the like, and the conventional CCD is used as it is.

Further, at the subsequent stage of the first CCD 12, C
A DS (Correlated Double Sampling) circuit 16, an A / D converter 17, and a DSP (digital signal processor) 18 for performing predetermined image processing are connected. The circuit 19, the A / D converter 20, and the DSP 21 are connected. Then, the outputs of the DSPs 18 and 21 are input and, for example, 1
Image synthesis circuit 23 that synthesizes an image corresponding to 600,000 pixels
Is provided, and a CPU 24 for controlling the circuits including the image synthesizing circuit 23 is provided.

FIG. 2 shows the internal structure of the image synthesizing circuit 23. As shown, the image synthesizing circuit 23 has a first memory for storing image data formed based on the output of the first CCD 12. 26, a second memory 27 for storing image data formed based on the output of the second CCD 13, and an addition circuit 28 for adding these image data.

In the pixels obtained by the CCDs 12 and 13, a missing pixel generating circuit 29 for generating data of missing pixels which are missing, an output of the missing pixel generating circuit 29 and an output of the adding circuit 28 are added, and all pixels are added. An addition circuit 30 for forming an image with information is arranged. The above-described missing pixel generation circuit 29 calculates the average value of the pixel information on the upper, lower, left, and right sides of the missing pixel or the average value of only the left and right pixels (may be another method) and outputs the result as a missing pixel signal.

FIGS. 4 and 5 show the first CCD 12.
And pixel information obtained by the second CCD 13.
FIG. 7A shows the first CCD 12 in an image (screen) area W.
(B) is the pixel data obtained by the second CCD 13, and FIG. (C) is the added image data. As described with reference to FIG. 3, the CCDs 12 and 13 are obliquely shifted from each other by one pixel of a (= b), thereby outputting image data at different pixel positions as shown in FIGS. Will do.

That is, as shown in FIG. 4A, a pixel A (11 × 1) conventionally set in the CCDs 12 and 13 is used.
The data of the pixels a1, a2, a3... having a size of 1/4 of l2) is obtained by the CCD 12, and as shown in FIG. 4B, the pixels b1, b2, b3. Is obtained by the CCD 13. In FIG.
For example, a state in which the character “All” is photographed is indicated, and CC is displayed.
In D12, the pixel data (black part) in FIG.
In 3, the pixel data (hatched portion) in FIG. An image signal composed of these pixel data is subjected to predetermined processing by the CDSs 16 and 19 and the DSPs 18 and 21 in FIG. 1 and then stored in the first and second memories 26 and 27 in the image synthesizing circuit 23. Is done.

In the image synthesizing circuit 23, the adding circuit 2
8, the image data in the first memory 26 and the image data in the second memory 27 are added to obtain the character "ALL" as shown in FIG. This FIG. 4 (C)
Is composed of 800,000 (400,000 × 2) pixels, and is an image having twice the resolution when one CCD having 400,000 pixels is used.

However, in the image shown in FIG. 4C, the size of the pixel is reduced to 1/4, while two CCs are used.
Since D12 and D13 are used, a missing pixel e occurs as understood from FIG. Therefore, in this example, the missing pixel generation circuit 29 is provided as described above, and the missing pixel e is processed from surrounding pixel data. That is, the screen W in FIG.
Considering the missing pixel e12, the arithmetic mean of the pixel data of the upper and lower a2 and a12 and the left and right b1 and b2 is calculated, and this is used as the pixel data of the missing pixel e12. In addition, when there is only one data at the top and bottom, left and right in the pixel at the end,
Correction such as doubling is performed. Various known methods can be applied to the method of processing the data of the missing pixels.

The missing pixel data is output from the missing pixel generating circuit 29 to the adding circuit 30 and is added to the output of the adding circuit 28, whereby one image having no missing pixels is obtained. Is formed. The image in this case is an image of 1.6 million (= 400,000 × 4) pixels. Therefore, in this example, two CCDs 1 of 400,000 pixels are used.
By using 2, 13, high-resolution image quality with four times the number of pixels can be obtained.

FIG. 6 shows another example of the configuration in the image synthesizing circuit 23. In this example, missing pixel data is generated using only left and right data.
In FIG. 6, a first memory 26 is connected to a first missing pixel generation circuit 32 and an addition circuit 33 which calculate an average of left and right data to generate missing pixel data, and a second memory 27.
Similarly, a second missing pixel generating circuit 34 and an adding circuit 35 are used to calculate the average of the left and right data to obtain missing pixel data.
Is provided.

In this example, a field image in which missing pixels on each line are filled is formed, and the output of the addition circuit 33 is used as image data of an odd field, and the output of the addition circuit 35 is used as image data of an even field. By using these field image data and performing interlaced scanning, an image can be displayed.

In the above example, the same image is picked up by the two CCDs 12 and 13. However, three or four CCDs can be used, and the number of separated optical paths can be increased by an optical path separating element to newly arrange CCDs. Thus, the resolution can be further improved.

[0022]

As described above, according to the present invention,
For example, an image sensor is arranged in each of the optical paths separated by a half mirror, and image signals composed of data at different pixel positions are extracted and combined, so that the number of pixels conventionally obtained by the used image sensor is used. For example, an image having four times the number of pixels can be formed, and a high-resolution image that can be used as a high-vision image or the like can be obtained.

According to the second aspect of the present invention, the image synthesis circuit calculates and generates a missing pixel other than the pixel set by the image pickup device from surrounding pixel signals, so that a missing pixel is generated. Even in this case, there is an advantage that a high-resolution image can be formed by satisfactorily satisfying the data of the missing pixel.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an image forming apparatus in which a plurality of image sensors are combined according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration in an image composition circuit of FIG. 1;

FIG. 3 is a diagram showing a mounting relationship between two CCDs in an image light path according to the embodiment;

FIG. 4 shows pixel information obtained by two CCDs according to the embodiment; FIG. 4A shows pixel data obtained by a first CCD;
FIG. (B) shows pixel data obtained by the second CCD, and FIG.
Is image data obtained by adding both.

FIG. 5 is a diagram illustrating a configuration of pixel data in an image according to the embodiment.

FIG. 6 is a block diagram showing another configuration in the image composition circuit of the embodiment.

[Explanation of symbols]

 10 Objective optical system, 11 Half mirror, 12, 13 CCD, 16, 19 CDS, 18, 21 DSP, 23 Image synthesis circuit, 24 CPU, 26, 27 Memory, 29, 32, 34 ... missing pixel generation circuit, 28, 30, 33, 35 ... addition circuit.

Claims (2)

[Claims] 1. An optical path separating element for separating an image receiving side optical path from one objective optical system, and an image signal composed of data at different pixel positions respectively arranged on the optical paths separated by the optical path separating element. A plurality of image sensors arranged so as to perform image processing and image signals obtained from the plurality of image sensors,
An image forming apparatus, comprising: an image combining circuit that forms an image having a larger number of pixels than the number of pixels of the image sensor. 2. The image pickup device according to claim 1, wherein the image synthesis circuit calculates a missing pixel other than the pixel set by the image pickup device from surrounding pixel signals and generates the missing pixel. Combined image forming device.