JP2003348607A - Imaging apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus and a method for more accurately correcting a defective pixel by means of adjacent pixels and decreasing an error due to a shift in an imaging device, so as to suppress deterioration in an image. <P>SOLUTION: A shift means 3 periodically shifts an imaging device 2 for imaging an object image and allows the imaging device 2 to output an image signal at each shifted position, an image processing section 4 rearranges the image signals of the imaging device 2 to composite images, and a defective pixel correction section 5 applies different correction processing to a defective pixel of the composed image responding a shifted position by the shift means 3. <P>COPYRIGHT: (C)2004,JPO

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image pickup apparatus and method such as a digital camera (electronic still camera) having a pixel defect correction function.

[0002]

2. Description of the Related Art It is generally known that a solid-state imaging device such as a CCD formed of a semiconductor deteriorates pixels due to local crystal defects or the like.

As a method of performing pixel defect correction for such pixel deterioration, for example, as disclosed in Japanese Patent Application Laid-Open No. 2000-59999, attention is paid to four adjacent pixels in the upper, lower, left and right directions around a defective pixel. A method has been considered in which defective pixels are compensated for by interpolating defective pixels with four pixels, and pixel deterioration is suppressed.

According to such a pixel defect correction method,
Since adjacent pixels usually have similar information,
The resolution is not significantly degraded by interpolation using these adjacent pixels, and an effective result is obtained.

[0005] In recent years, imaging devices such as digital cameras (electronic still cameras) using a two-dimensional solid-state imaging device have become widespread. However, in such an imaging device, an image having a higher resolution is required. It has become required.

Therefore, as a means for obtaining a high resolution, for example, as disclosed in Japanese Patent Application Laid-Open No. 8-251604, a pixel shift method of displacing an image pickup element in a two-dimensional direction to pick up an image and combining them is provided. By adopting,
A method for obtaining a high-definition image has been used.

However, with respect to such a pixel shifting method, if a method of interpolating a defective pixel by using the above-described four pixels adjacent to the upper, lower, left, and right sides of the defective pixel and correcting the defective pixel is adopted, the defective pixel is replaced with the adjacent defective pixel. Since the interpolation is performed by itself, it is difficult to suppress the deterioration of the image.

To solve such a problem, for example, as disclosed in Japanese Patent Application Laid-Open No. Hei 7-322151, by performing pixel shift at a large interval separated by at least one pixel, a defective pixel is added to a synthesized image. Are not placed adjacent to each other,
At the time of defective pixel compensation, a defective pixel correction method has been conceived, in which interpolation is not always performed by adjacent defective pixels themselves, and deterioration of an image can be suppressed by interpolation using normal pixels.

[0009]

However, although increasing the amount of displacement of the image sensor at the time of imaging can increase the effect of compensating for defective pixels, it requires moving the image sensor one or more pixels away. However, there is a possibility that a moving error of the image sensor becomes large and causes image deterioration. In addition, a piezoelectric element such as a piezo element is used to displace the imaging element.
Since the driving voltage needs to be increased, there is also a problem that the driving circuit becomes large.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and provides an image pickup apparatus capable of more accurately performing defective pixel correction by adjacent pixels, reducing errors due to displacement of an image pickup device, and suppressing image deterioration. It is in.

[0011]

According to the first aspect of the present invention,
An image sensor that captures a subject image, a displacement unit that periodically displaces the image sensor and outputs an image signal of the subject image from the image sensor for each displacement position, and for each displacement position by the displacement unit Image processing means for receiving the image signal of the image sensor and rearranging the image signal to synthesize an image, and an image from a defective pixel of the image sensor among image data synthesized by the image processing means. Defective pixel correcting means for performing different correction processing on the image data synthesized based on the signal in accordance with the displacement position by the displacement means.

According to a second aspect of the present invention, in the first aspect of the present invention, the displacement means includes a 3 × 3 pixel at an interval of 2/3 pixels.
The position of the image sensor is displaced to a position to shift a pixel of an image signal output from the image sensor, and the defective pixel correction unit is different depending on a pixel shift position of the image signal output from the image sensor. In this case, a defective pixel correction value is obtained based on the image signal at the position of.

According to a third aspect of the present invention, in the second aspect of the present invention, the defective pixel correcting means is arranged so that the middle pixel of the pixel shift positions of the image signal output from the image pickup device is tilted in the left, right, up and down directions. A correction value obtained by multiplying the sum of the signal levels of the four pixels of the same color in the upper, lower, left and right directions by 1/2 for the defective pixel located in A correction value obtained by adding the signal levels of the four pixels of the same color in the upper, lower, left and right directions is obtained, and the sum of the signal levels of two pixels of the same color located at a distance of four pixels between the left and right pixels with respect to the defective pixel located in the middle is obtained Are calculated by multiplying the correction values by the correction factor, and correction processing of the corresponding defective images is performed using the correction values.

According to a fourth aspect of the present invention, in the first aspect of the present invention, the image pickup device has a color filter for acquiring color discrimination information, and the defective pixel correction unit is provided with the image processing unit. Different correction processing is performed on defective pixels of an image synthesized by the above according to the color determination information acquired by the color filter and the displacement position by the displacement means.

According to a fifth aspect of the present invention, in the fourth aspect of the invention, when the defective pixel is determined to be G (green) based on the color discrimination information, the defective pixel correction means The sum of the signal levels of four pixels of the same color
A correction value obtained by multiplying by / 4 is obtained, and a correction process is performed using the correction value.

According to a sixth aspect of the present invention, in the first aspect of the present invention, the defective pixel correction means further includes a position information conversion means, and the position information conversion means converts the position information of the image signal after the pixel shift. Is converted into a plurality of image signals, and these divided image signals are sequentially processed.

According to a seventh aspect of the present invention, in the first aspect of the invention, there is provided a microscope for projecting a subject image onto the image pickup device, and a light shielding means for selecting whether or not to project the subject image onto the image pickup device. A defective pixel detecting unit that detects a defect of each pixel of an image signal output from the imaging element in a state where the projection of the subject image onto the imaging element is shielded by the light shielding unit; and the defective pixel detecting unit. Defective pixel correction means for performing a correction process on the defective pixel detected by the method.

According to the present invention, an image sensor for picking up a subject image is periodically displaced by a displacement means, and the image sensor outputs an image signal of the subject image for each displacement position. An imaging method for rearranging image signals for each displacement position to synthesize an image, wherein image data synthesized based on an image signal from a defective pixel of the image sensor among the image data to be synthesized. A different correction process is performed according to the displacement position by the displacement means.

As a result, according to the present invention, it is possible to perform more accurate defective pixel correction by making the correction processing of the defective pixel different according to the pixel shift position, and the width of the pixel shift is one pixel. Since it can be reduced to within, image degradation due to an error due to displacement of the image sensor can be suppressed at the same time.

Further, according to the present invention, a defective pixel of G (green) can be corrected by a normal adjacent pixel of the same color, so that a defective pixel of G (green) which greatly affects the resolution of an image can be corrected. And accurate correction can be realized.

Furthermore, according to the present invention, since the width of one line can be reduced by dividing the image signal, equivalent processing can be performed using a small capacity.

Furthermore, according to the present invention, since the correction processing can be performed also on pixels where dark noise is conspicuous, it is possible to reduce image noise in the case of photographing a dark sample such as fluorescence observation with a microscope. Can be.

[0023]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 shows a schematic configuration of an imaging apparatus to which the present invention is applied. In FIG. 1, reference numeral 1 denotes an imaging lens, and a subject image passing through the imaging lens is formed on an imaging surface of an imaging device 2 formed of a CCD. The imaging element 2 photoelectrically converts the subject image formed on the imaging surface and outputs the analog image signal to the image processing unit 4 of the image processing unit.

A displacement means 3 is connected to the image pickup device 2. The displacement means 3 has a piezoelectric element such as a piezo element. The image sensor 2 is operated by the operation of the piezoelectric element.
It is periodically displaced and outputs an image signal of a subject image for each displacement position. As a result, an image signal for each displacement position of the image sensor 2 is input to the image processing unit 4.

The image processing section 4 reads out and rearranges image signals for each displacement position of the subject, and synthesizes them into one image. The image processing unit 4 is connected to a defective pixel correction unit 5 as a defective pixel correction unit. This defective pixel correction unit 5
The correction unit 3 corrects a defective pixel in the image synthesized by the image processing unit 4 according to the displacement position of the displacement unit 3.

FIG. 2 is a diagram for explaining the imaging apparatus thus configured in more detail, and the same parts as those in FIG. 1 are denoted by the same reference numerals.

In this case, the displacement means 3 is composed of a piezoelectric element 31 such as a piezo element and a piezoelectric element driver 32 for driving the piezoelectric element 31. Are periodically displaced, and an image signal of a subject image at each displacement position is output.

As shown in FIG. 3, the image sensor 2 here is driven by a piezoelectric element driver 32 by a piezoelectric element driver 32 at intervals of 2/3 pixels with respect to the reference pixel position.
→→… →→ 3 in the vertical and horizontal directions
Pixels are shifted at a total of nine (3 × 3) locations, and an image signal of a subject image is output for each of these displacement positions. FIGS. 4A and 4B show the color arrangement of the pixels before and after the pixel shift. FIG. 4A shows the color arrangement of the R, G, and B pixels before the pixel shift shown in FIG. The image after nine pixel shifts at 2/3 pixel intervals according to the procedure shown in FIG. 3 is as shown in FIG. 3B, and the color arrangement of R, G, B is unchanged, The number of pixels is about three times as large in the vertical direction.

Returning to FIG. 2, the image processing section 4 comprises a solid-state image sensor driver 41, an A / D converter 42, a memory 43, a defective ROM 44, and a CPU 45.

The solid-state image sensor driver 41 includes the image sensor 2
Is driven with an exposure time based on a predetermined timing signal,
The object image is output as an analog image signal. The A / D converter 42 converts an analog image signal output from the image sensor 2 into a digital image signal. The memory 43 stores an image signal digitized by the control signal from the CPU 45 through the A / D converter 42 for each pixel shift position. Defect R
The address of the defective pixel position unique to the image sensor 2 is stored in the OM 44 in advance. Further, the CPU 45 reads and rearranges the image signal stored in the memory 43 for each pixel shift position for each pixel, synthesizes the image as one image, and then outputs the image signal, the displacement position signal indicating the pixel shift position, and the pixel position. The address (position information) is output to the defective pixel correction unit 5.

The defective pixel correction section 5 comprises a RAM 51, a defective address detection section 52, and a defect compensation circuit 53.

The RAM 51 is a defective ROM of the image processing section 4.
The address of the defective pixel position unique to the image sensor 2 stored in advance in the CPU 44, that is, the defective pixel address is stored in the CPU.
45. The defective address detection unit 52 calculates a defective pixel address stored in the RAM 51,
The image signal input through the CPU 45 of the image processing unit 4 is compared with the address (position information) of each pixel, and when they match, it is determined that a defective pixel has been detected. Output to The defect compensation circuit 53 receives a displacement position signal indicating a pixel displacement position of the image signal together with the image signal through the CPU 45 of the image processing unit 4 and does not detect a defect detection signal from the defect address detection unit 52. Outputs an image signal without performing any processing, and when a defect detection signal is detected, a defective pixel is corrected by processing according to the displacement position signal.

Next, a process of correcting a defective pixel performed by the defect compensation circuit 53 will be described.

FIGS. 5 and 6 show a schematic configuration of the defect compensation circuit 53. FIG. The defective pixel mask unit 55 shown in FIG. 5 replaces the gradation of the pixel data with 0 when it determines that the input image data is a defective pixel based on the defect detection signal Y input from the defective address detection unit 52. Output. Accordingly, this causes all the defective pixels in the subsequent circuit to have the gradation 0. In FIG. 5, (H) is a line buffer, and (D) is a D-FF. Each of the line buffer (H) and the D-FF (D) has a CLK terminal.
(Not shown), and transfers an image signal every 1 CLK.

Then, for the image signal inputted one pixel at a time in synchronization with the CLK, the line buffer (H) is output once to four times.
To generate pixel signals B1, C1, D1, and E1 delayed by 1 to 4 lines with respect to the pixel signal A1. Further, these pixel signals A1, B1, C1, D1, E
1 through the D-FF (D) a plurality of times, A1 to A delayed from the image signal to 4 lines +8 CLK
9, B1 to B9, C1 to C9, D1 to D9, E1 to E9
Is generated. By doing so, it is possible to refer to an arbitrary image signal in a 5 × 9 pixel range in the same time zone, and to set the level of each pixel signal for Ai, Bi, Ci, Di, and Ei (i = 1 to 9). Observable.

FIG. 6 is a block diagram of a circuit for correcting a defective pixel in the defect compensation circuit 53. Symbols C5, A5, C3, C7, E5, C1, and C9 in the drawing represent pixel signals in the 5 × 9 pixel range shown in FIG. Among them, the pixel signal C5 is directly input to the output selection section 531. The pixel signals A5, C3, C7, E5 are
The sum output of the adder 532 is input to the output selection unit 531 as a correction signal A via a 乗 算 multiplier 533, and the output of the adder 532 is output as a correction signal B. It is directly input to the output selection section 531. Further, the pixel signals C1 and C9 are input to an adder 534, and the sum output of the adder 534 is a 乗 算 multiplier 53.
5 is input to the output selection section 531 as a correction signal C. As a result, assuming that the pixel signal C5 corresponds to the central pixel, the correction signal A includes four pixel signals A5, C3, C7, E of the same color in the upper, lower, left, and right colors as shown in FIG.
The correction signal B is equal to the value obtained by multiplying the sum of the signal levels of H.5 by 1/2, and the correction signal B is the signal level of the pixel signals A5, C3, C7, E5 of the four pixels of the same color in the upper, lower, left, and right colors as shown in FIG. 7C, and the correction signal C is に of the sum of the signal levels of the pixel signals C1 and C9 of two pixels of the same color that are located at a distance of 4 pixels between the left and right as shown in FIG. 7C. Multiplied by.

The output selection unit 531 includes the C of the image processing unit 4.
A displacement position signal X input through the PU 45 and a defect detection signal Y input from the defect address detector 52 are provided. When the defect detection signal Y is not supplied, the output selection unit 531 outputs the pixel signal C5 as it is. Also, when the defect detection signal Y is given, the correction signals A, B, C according to the type of the displacement position signal X input at this time.
The pixel correction process is performed by either of the above. That is, the output selection unit 531 has the table shown in FIG.
When the displacement position signal X is the pixel shift position shown in FIG.
Is selected, a correction signal A is selected if the pixel corresponds to the pixel shift position, and a correction signal C is selected if the pixel corresponds to the pixel shift position. Correction processing of a defective pixel is performed by the correction signals A, B, and C.

Next, the defective pixel correction by the defect compensation circuit 53 will be described in more detail. FIG. 9 (a)
(B) and (c) show the pixels after the pixel shift described in FIG. 4 extracted for each of R, B, and G colors. Here, paying attention to the B (blue) pixel shown in FIG. 9B, if one of the pixels has a defect, the pixel is shifted by nine pixels as shown in FIG. Pixel B
(Blue) will contain defects.

FIG. 11 shows a method of correcting a defective pixel which is conventionally used. According to this method, as shown in FIG. 4 pixels B1 to B4 of the same color adjacent vertically and horizontally
To correct. As a result, for example, when the pixel of interest B0 is located at the upper left of nine defects as shown in FIG. 4B, pixels B3 and B4 of four pixels B1 to B4 of the same color vertically and horizontally adjacent to each other are defective pixels ( The pixel B0 of interest is included as shown in FIG.
Is located in the middle of nine defects, all of the four pixels B1 to B4 of the same color that are vertically and horizontally adjacent are defective pixels (hatched portions in the figure). For this reason, if the defective pixel correction processing is executed by these methods, the processing results differ depending on the defect position, and it is difficult to expect a uniform effect.

On the other hand, in the present invention, when the displacement position signal X corresponds to the pixel shift position,... Shown in FIG. 3, two pixels out of four adjacent pixels of the same color as shown in FIG. Defective pixel (with output gradation of 0) (indicated by hatching)
, The correction signal A (correction value) becomes equal to the average value of two normal pixels of the same color adjacent to each other. For example, at the pixel shift position, among the pixel signals A5, C3, C7, and E5 of four pixels of the same color in the upper, lower, left, and right directions shown in FIG.
Since 5 is a defective pixel and the output gradation becomes 0, the correction signal A (correction value) is a pixel signal A of two normal adjacent pixels of the same color.
5, calculated as the average value of C3. Pixel shift position
Similarly, the correction signal A (correction value) is
It is obtained as an average value of two normal adjacent pixels of the same color.

If the displacement position signal X corresponds to the pixel shift position ,,, shown in FIG. 3, as shown in FIG. 10, three out of four adjacent pixels of the same color (output gradation of 0) are defective. Since pixels (indicated by hatching) are included, the correction signal B
The (correction value) is equal to the value of one normal adjacent pixel of the same color. For example, at the pixel shift position, pixel signals A5, C3, C7, E of four pixels of the same color in the upper, lower, left, and right colors shown in FIG.
5, the pixel signal C3, C7, E5 becomes a defective pixel and the output gradation becomes 0, so that the correction signal B (correction value) is obtained as the value of the pixel signal A5 of one normal adjacent pixel of the same color. Similarly, the correction signal B (correction value) is obtained as the value of one normal adjacent pixel of the same color.

Further, when the displacement position signal X corresponds to the pixel shift position shown in FIG. 3, as shown in FIG. 10, all four adjacent pixels of the same color are defective pixels (the output gradation is 0) Therefore, the correction signal C (correction value) is
It is equal to the average value of the nearest normal two pixels of the same color. For example, at the pixel shift position, the pixel signals A5, C3, C7, and E5 of the four pixels of the same color in the upper, lower, left, and right directions shown in FIG. 7C all become defective pixels and the output gradation becomes 0, so the correction signal C ( The correction value) is obtained as the average value of the pixel signals C1 and C9 of the two nearest normal pixels of the same color.

After that, if the correction of the defective pixel is performed by using the different correction signals A, B, and C according to the pixel shift position, the defective pixel is replaced with the pixel signal of the same color at any displacement position. Can be corrected uniformly.

Accordingly, this makes it possible to perform more accurate defective pixel correction by making the correction process for defective pixels different according to the pixel shift position. Further, since the width of the pixel shift is as small as one pixel or less, the error due to the displacement of the image sensor can be reduced, and the image degradation due to the displacement error can be suppressed at the same time.

In this embodiment, when the pixel signals of the same color in the upper, lower, left, and right colors are all defective pixels, the correction value is obtained by normal pixels of the same color in the vicinity of the left and right. It is also possible to obtain by using neighboring normal same-color pixels. In addition, by making the number of delay elements (D-FF) variable simultaneously with the line buffer, a correction value based on an image signal at an arbitrary position can be obtained.

(Second Embodiment) Next, a second embodiment of the present invention will be described.
An embodiment will be described.

The feature of the second embodiment is that a color filter is provided and a defective pixel correction method is changed for each color.

FIG. 12 shows a schematic configuration of the second embodiment of the present invention. The same parts as those in FIG. 2 are denoted by the same reference numerals.

In this case, the image sensor 2 is provided with an array of color filters 6 for acquiring color discrimination information. Others are the same as FIG.

In such a configuration, the subject image formed on the image sensor 2 through the color filter 6 is
After the photoelectric conversion, the image data is input to the image processing unit 4. In this case, the image sensor 2 is vertically and horizontally shifted by three pixels in the vertical and horizontal directions by the piezoelectric element 31 driven by the piezoelectric element driver 32, for a total of nine pixel shifts. Is input to the image processing unit 4 together with the discrimination information.

The image processing section 4 synthesizes a single image based on the image signal input from the image sensor 2, and then outputs the image signal, the displacement position signal of the image signal, the address (position information) of each pixel, and the color discrimination. The information is output to the defect compensation circuit 53.

The defect compensating circuit 53 has A1 to A9, B1 to B9, C1 to C9, and D1 to D9 delayed by 4 lines +8 CLK with respect to the image signal by the circuit configuration described with reference to FIG.
9. Generate pixel signals E1 to E9.

FIG. 13 is a block diagram of a circuit for correcting a defective pixel in the defect compensation circuit 53. Reference symbols C5, A5, C3, C7, E5, C1, C9, B4,
B6, D4, and D6 represent pixel signals in the 5 × 9 pixel range shown in FIG. Of these, the pixel signals C5 and A
For C5, C3, C7, E5, C1, and C9, a circuit for generating correction signals A, B, and C similar to that described with reference to FIG. 6 is configured. Further, the pixel signals B4, B6, D
4, D6 are input to the adder 536, and the adder 53
The sum output of 6 is input to the output selection unit 531 as a correction signal D via the 1/4 multiplier 537. As a result, assuming that the pixel signal C5 corresponds to the center pixel, the correction signal D becomes the sum of the signal levels of the four pixel signals B4, B6, D4, and D6 of the same color in the left, right, upper, and lower directions as shown in FIG. 1
/ 4.

The output selection section 531 includes the image processing section 4
Position signal X input from the controller and the defect address detector 5
2, a color discrimination signal Z input from the image processing unit 4 is provided.
When the defect detection signal Y is not supplied, the output selection unit 531 outputs the pixel signal C5 as it is. Further, when the defect detection signal Y is given, the correction signal A, the correction signal A,
The pixel correction process is performed by one of B, C, and D. That is, the output selection unit 531 prepares the table shown in FIG. 15, and when the color discrimination signal Z is R (red) or B (blue), the displacement position signal X becomes the pixel shift position shown in FIG.
,,, the correction signal A,
The correction signal B is selected when the pixel shift position corresponds to the pixel shift position, and the correction signal C is selected when the pixel shift position corresponds to the pixel shift position. Selects the correction signal D, and performs correction processing of the defective pixel by using the correction signals A, B, C, and D.

In this case, the method of correcting the defective pixel when the color discrimination signal Z is R (red) or B (blue) is the same as in the first embodiment. That is, the displacement position signal X is
In the case of the pixel shift position,
As shown in FIG. 10, two of the four adjacent pixels of the same color include a defective pixel (shown by a shaded portion) (having an output gradation of 0), so that the correction signal A (correction value) is a normal two adjacent pixels of the same color. Is calculated as the average value of In addition, when the displacement position signal X corresponds to the pixel shift position ,,,, shown in FIG. 3, as shown in FIG. 10, three out of four adjacent pixels of the same color have a defective pixel (having an output gradation of 0) (a hatched line). The correction signal B (correction value) is obtained as a normal value of one adjacent pixel of the same color. Further, when the displacement position signal X corresponds to the pixel shift position shown in FIG. 3, as shown in FIG. 10, all the four adjacent pixels of the same color are defective pixels (shown by hatching) with an output gradation of 0. Therefore, the correction signal C (correction value) is obtained as an average value of the nearest two pixels of the same normal color.

On the other hand, the method of correcting a defective pixel when the color discrimination signal Z is G (green) is different from that of the first embodiment. FIG.
Represents the pixels after the pixel shift described in FIG.
9C are extracted for each of the colors B and G. Focusing on the G pixel shown in FIG. 9C, FIG.
The number of pixels is larger than that of the R and B pixels shown in (b), and the G pixels of the same color after the pixel shift are present at obliquely adjacent positions.

Therefore, when the color discrimination signal Z is G (green),
Regardless of the displacement position signal X, the correction signal D is converted to pixel signals B4, B
A value obtained by multiplying the sum of the signal levels of D6, D4, and D6 by 1/4 is used, and the correction processing of the defective pixel is performed using the correction signal D.

Accordingly, in this manner, in the pixel shifting method in which the pixel is displaced to 9 places at an interval of 2/3 pixels, G
(Green) defective pixels can be corrected by normal adjacent pixels of the same color, so that accurate correction can be realized for G (green) defective pixels that greatly affect the resolution of an image.
A reduction in image resolution can be suppressed.

(Third Embodiment) Next, a third embodiment of the present invention will be described.
An embodiment will be described.

The feature of the third embodiment is that the image data is divided by converting the address of the image after the pixel shift outputted from the image processing section, and the defective pixels are corrected sequentially for each divided image signal. Is to do.

FIG. 16 shows a schematic configuration of the third embodiment of the present invention. The same parts as those in FIG. 2 are denoted by the same reference numerals.

In this case, the defective pixel correction section 5 is provided with an address conversion section 54 as position information conversion means. The address conversion unit 54 divides the image signal into a plurality by converting the image signal address after the pixel shift and the defective pixel address based on the table stored in the defective ROM 44 while transmitting and receiving data to and from the CPU 45. The divided image signal and address are sequentially subjected to a defect compensation circuit 5
3 is output. Others are the same as FIG.

In such a configuration, the subject image formed on the image pickup device 2 through the lens 1 is photoelectrically converted by the image pickup device 2, and furthermore, three points in the vertical and horizontal directions by the piezoelectric element 31 for a total of 9 points. After the pixel is shifted to the location, the image is input to the image processing unit 4. Then, the image processing unit 4
After synthesizing one image by the same procedure as described in the first embodiment, the image signal, the displacement position signal of the image signal, and the address (position information) of each pixel are converted to an address conversion unit 54.
Output to

The address conversion unit 54 changes the image signal address after the pixel shift and the defective pixel address in accordance with various tables stored in the defective ROM 44 while transmitting and receiving data to and from the CPU 45, and divides the image signal into a plurality of image signals. Then, the divided image signals and addresses are sequentially output to the defect compensation circuit 53. The defect compensation circuit 53 corrects a defective pixel in the same procedure as described in the first embodiment based on the divided image signal input from the address conversion unit 54 and its address, and outputs the result. .

Here, the procedure of address conversion in the address conversion section 54 will be described in detail. FIG. 17 is an image having the number of pixels of 2Ya in the vertical direction × 2Xa in the parallel direction after pixel shifting, and addresses are assigned to (0 to 2Xa, 0 to 2Ya).

First, a description will be given of a case where this image signal is divided into four images having a height Xa and a width Ya as shown in FIG.

FIG. 20 is a flow chart showing the procedure of the address conversion process.
1, the position (x, y) of the input pixel is
It is determined which of the two areas belongs to based on the table shown in FIG. Here, for example, x <Xa, y <Y
If it is a, it is determined that the pixel position exists in the area. Next, in step 202, the image signal is converted for each area based on the table shown in FIG. That is, when it is determined that the pixel is in the area, the address is output without conversion. When it is determined that the pixel is in the area, the vertical address y is converted to y '= y-Ya without changing the horizontal address x. In the case of a region, conversion is performed in the same manner.

For example, (Xa, Ya) = (1000, 10
00), the address of the pixel (x, y) = (1200, 1500) is determined to be in the area since Xa <1200 and Ya <1500, and the address is (1200-1).
000,1500-1000) = (200,500).

As described above, the address of the image signal is obtained by executing the above-described processing, as shown in FIG. 19, by setting the number of horizontal pixels Xa and the number of vertical pixels Ya with the upper left end of each divided area at (0, 0). Is converted to the address of the area.

Next, a procedure for converting defective pixel addresses in order to perform defect correction on the divided image signals will be described.

In this case, the defective pixel exists uniquely for each imaging element 2, and the defective pixel address is represented as position information before the pixel is shifted. Therefore, in order to perform the defective pixel correction process on the image after the pixel shift, the defective pixel address also needs to be converted.

Here, it is assumed that the number of pixels of the image sensor 2 is 2Xa 'and 2Ya' in the horizontal and vertical directions, respectively.

FIG. 21 is a flowchart showing a processing procedure for converting the defective pixel address stored in the defective ROM 44. First, in step 211, the defective pixel (x, y) is divided into the divided image of the entire image. Determine if it exists in the area. Next, in step 212, address conversion is performed for each area based on the table. The processing procedure up to address conversion for each area is as follows: Xa → Xa ′, Ya
By replacing with Ya ′, it is the same as the case of the address conversion of the image signal described above.

Next, in step 213, processing for coping with pixel shift is performed. In this case, when the pixel is shifted to 9 points as in the above-described embodiment, the resolution (the number of pixels) is tripled in the horizontal and vertical directions, respectively. The address of the area 1 to 3 is a value that is tripled.

Then, in step 214, an address corresponding to the pixel shift position is added. In this case, if the defective pixel is displaced to 9 points by 2/3 pixel as shown in FIG. 3, all of these positions become defective pixels. Therefore, the address conversion formula of the table shown in FIG.
Are substituted, and the calculated eight addresses are newly added as defective pixel addresses and stored in the RAM 51 of the defective pixel correction unit 5.

Through such a series of processing, the image after the pixel shift is also divided into four areas of width Xa and height Ya, and both the image signal and the defective pixel address are 0 to Xa, 0 to 0.
It is converted to the range of Ya. That is, the widths of one line and one frame are replaced with Xa and Ya, respectively.

The address-converted image signal and defective pixel address are sequentially input to the defect compensation circuit 53. In this case, the defect compensation circuit 53
a, the image signal of the height 2Ya is not inputted at a time, but the converted image signals of the addresses 0 to Xa, 0 to Ya are sequentially inputted in four times.

Therefore, with this configuration, the line buffer (H) in the defect compensation circuit 53 shown in FIG. 5 can be reduced in capacity by reducing the width of one line by dividing the image signal. However, the same processing can be performed. In other words, according to the present embodiment, the high-resolution image signal after the pixel shift is divided, and the image defect compensation processing is sequentially performed for each of the divided images. Can be realized with a low-capacity memory.

In this embodiment, an example has been described in which the present invention is applied to 9-point pixel shift of 2/3 pixel shift. However, the present embodiment is applied to an image pickup apparatus having a pixel shift mechanism having different pixel shift amounts and times. Is also possible. Also, the number of divisions of the image signal can be arbitrarily changed.

(Fourth Embodiment) Next, a fourth embodiment of the present invention will be described.
An example will be described.

The feature of the fourth embodiment resides in that the image pickup device is mounted on a microscope, and a pixel having a large dark current is detected and corrected.

FIG. 27 shows a schematic configuration of the fourth embodiment of the present invention. The same parts as those in FIG. 2 are denoted by the same reference numerals.

In this case, the imaging device 100 of the present invention includes:
The microscope 200 is connected. The microscope 200 has a sample 11 to be a subject mounted on a stage 12.
Light emitted from a light source 13 such as a halogen lamp disposed below the stage 12 is applied to the sample 11 by the illumination optical system 1.
4. Irradiate through condenser lens 15.

The image formed by the light transmitted and scattered through the sample 11 is
The image is enlarged by the objective lens 16 and projected on the imaging surface of the imaging device 2 by the projection lens 17.

A shutter 18 as a light shielding means is arranged on the front surface of the image pickup surface of the image pickup device 2. A shutter driver 19 is connected to the shutter 18.
The shutter driver 19 is driven according to a signal transmitted from the CPU 45 to open and close the shutter 18.

The image pickup device 2 periodically shifts pixels at nine positions in three positions in the vertical and horizontal directions by the piezoelectric element 31 driven by the piezoelectric element driver 32, and the image of the subject image at these displacement positions is obtained. Output a signal.

The image signal output from the image sensor 2 is A
The signal is converted into a digital image signal by the / D converter 42 and input to the memory 43. The output signal (digital image signal) of the A / D converter 42 is also input to a defective pixel detection circuit 20 as a defective pixel detection unit.

The defective pixel detection circuit 20 has a predetermined level (TH) of luminance set in advance,
When a dark current of a level exceeding (TH) is detected, the pixel position (address) is determined via the CPU 45 via the defective pixel correction unit 5.
In the RAM 51.

The other points are the same as those in FIG.

In such a configuration, when the observer turns on the power of the imaging apparatus 100, the CPU 45 operates the solid-state imaging device driver 4 with the shutter 18 closed.
1 and a signal to the piezoelectric element driver 32, and the
Is displaced to perform pixel shift at nine positions in total in three places in the vertical and horizontal directions, respectively, and an image signal of a subject image at these displacement positions is input to the A / D converter 42.

The A / D converter 42 converts the input image signal into a digital signal, and then converts the image signal into digital signals at each pixel position (1), (2),... The signal is input to the defective pixel detection circuit 20 as a luminance signal. The output signal of the A / D converter 42 at this time is
Is in the closed state and is output from the image sensor 2,
This is the dark current level of the image sensor 2.

As shown in FIG. 28, the defective pixel detection circuit 20 sets a predetermined level (TH) of luminance in advance, and when detecting a dark current having a luminance level exceeding the predetermined level (TH), The pixel position (address) is CP
The data is stored in the RAM 51 of the defective pixel correction unit 5 via U45. In this case, in FIG. 28, pixel positions (2) and (3)
(7) The brightness level of the dark current at (10) is a predetermined level
(TH), and these pixel positions (addresses) are
It is stored in the RAM 51 of the defective pixel correction unit 5 via the PU 45.

As described above, in the defective pixel detection circuit 20, the image signal (dark current) of all the pixel regions of the image sensor 2 is obtained.
, The detection of the luminance level is performed. Then the CPU
Reference numeral 45 also stores the defective pixel address stored in the defective ROM 44 in advance in the RAM 51.

The CPU 45 sends a signal to the shutter driver 19 to open the shutter 18. When the shutter 18 is opened, an image of the sample 11 from the microscope 200 is projected on the image sensor 2 immediately after the shutter 18 is opened.

Further, a signal is transmitted from the CPU 45 to the solid-state image sensor driver 41 and the piezoelectric element driver 32, and the image sensor 2 is displaced to perform pixel shift at nine positions in three places in the vertical and horizontal directions respectively. After digitizing the image signal at the position via the A / D converter 42,
3 is stored.

In this state, the CPU 45 does not operate the defective pixel detection circuit 20.

In the defective address detecting section 52, the memory 43
Is compared with the address stored in the RAM 51, and when they match, a detection signal is transmitted to the defect compensation circuit 53. The defect compensation circuit 53 outputs a corrected image signal when receiving the detection signal of the defect address detection unit 52, and outputs the image signal input from the memory 43 as it is when not receiving the detection signal. . The correction processing in the defect compensation circuit 53 here is the same as in the above-described first embodiment.

In this case, the defective ROM 44 is stored in the RAM 51.
In addition to the defective pixels stored in advance, the addresses of the pixels for which the dark current is determined to be higher than the predetermined level by the defective pixel detection circuit 20 are also stored.

As a result, the correction processing in the defect compensation circuit 53 is performed not only for defective pixels but also for pixels in which dark noise is conspicuous.
In the case where a dark sample 11 is photographed, for example, for fluorescence observation, image noise can be significantly reduced.

In the present embodiment, a pixel having a large dark current is detected when the power is turned on. However, the detection conditions are variable, and a pixel to be corrected is detected at predetermined time intervals during imaging and stored in the RAM 51. If the address is rewritten, it is possible to correct not only a dark current that changes over time but also a moving image noise such as a greeting. Also,
The light-blocking means for the image pickup device 2 is not limited to the shutter 18, but can also be realized by on / off control of the light source 13 itself and insertion of an optical filter or the like. Furthermore, a defective pixel can be detected during imaging by giving a defective pixel detection condition to the defective pixel detection circuit 20 without storing the position information in the defective ROM 44 in advance. At this time,
It is no longer necessary to change the contents of the defective ROM 44 for defective pixels unique to the image sensor 2, and it is possible to correct even defective pixels that change over time.

In addition, the present invention is not limited to the above-described embodiment, and can be variously modified in the implementation stage without departing from the spirit of the invention.

Furthermore, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent requirements. For example, even if some components are deleted from all the components shown in the embodiments, the problem described in the column of the problem to be solved by the invention can be solved, and the problem described in the column of the effect of the invention can be solved. In the case where the effect described above is obtained, a configuration from which this configuration requirement is deleted can be extracted as an invention.

[0104]

As described above, according to the present invention, there is provided an imaging apparatus and method capable of more accurately performing defective pixel correction by adjacent pixels, reducing errors due to displacement of the imaging element, and suppressing image deterioration. Can be provided.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating a schematic configuration of an imaging device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a schematic configuration for explaining the imaging device of the first embodiment in further detail;

FIG. 3 is a view showing a pixel displacement state by the pixel shifting method according to the first embodiment;

FIG. 4 is a diagram illustrating a color arrangement of pixels before and after pixel shift according to the first embodiment.

FIG. 5 is a diagram illustrating a schematic configuration of a defective pixel compensation circuit according to the first embodiment;

FIG. 6 is a block diagram of a circuit that performs interpolation of a defective pixel in the defect compensation circuit according to the first embodiment.

FIG. 7 is an exemplary view for explaining a defect correction method for each pixel shift position of a defective pixel according to the first embodiment;

FIG. 8 is a diagram illustrating a table prepared in an output selection unit according to the first embodiment.

FIG. 9 is a diagram illustrating a pixel arrangement when pixels after pixel shift according to the first embodiment are extracted for each of R, B, and G colors;

FIG. 10 is a diagram illustrating an example of the arrangement of defective pixels after pixel shifting according to the first embodiment;

FIG. 11 is a view for explaining a conventionally used defective pixel correction method as a comparative example of the first embodiment.

FIG. 12 is a diagram showing a schematic configuration of a second embodiment of the present invention.

FIG. 13 is a block diagram of a circuit for correcting a defective pixel in the defect compensation circuit according to the second embodiment.

FIG. 14 is a diagram for explaining a defective pixel correction method according to the second embodiment.

FIG. 15 is a diagram illustrating a table prepared in an output selection unit according to the second embodiment.

FIG. 16 is a diagram showing a schematic configuration of a third embodiment of the present invention.

FIG. 17 is a view for explaining an address conversion procedure in an address conversion unit according to the third embodiment;

FIG. 18 is a view for explaining an address conversion procedure in an address conversion unit according to the third embodiment;

FIG. 19 is a view for explaining an address conversion procedure in an address conversion unit according to the third embodiment;

FIG. 20 is a flowchart illustrating a procedure of address conversion in an address conversion unit according to the third embodiment;

FIG. 21 is a flowchart for explaining an address conversion procedure in an address conversion unit according to the third embodiment;

FIG. 22 is a view showing an example of a table stored in a defective ROM according to the third embodiment.

FIG. 23 is a view showing an example of a table stored in a defective ROM according to the third embodiment;

FIG. 24 is a diagram showing an example of a table stored in a defective ROM according to the third embodiment.

FIG. 25 is a diagram showing an example of a table stored in a defective ROM according to the third embodiment.

FIG. 26 is a diagram showing an example of a table stored in a defective ROM according to the third embodiment.

FIG. 27 is a diagram showing a schematic configuration of a fourth embodiment of the present invention.

FIG. 28 is a diagram illustrating a relationship between a predetermined level (TH) set in a defective pixel detection circuit according to a fourth embodiment and a luminance level of an image signal.

[Explanation of symbols]

1 ... Lens 2. Image sensor 3. Displacement means 31 ... Piezoelectric element 32 ... Piezoelectric element driver 4: Image processing unit 41 ... Solid-state image sensor driver 42 ... A / D converter 43… Memory 44 ... Defective ROM 45 ... CPU 5. Defective pixel correction unit 6 ... Color filter 11 ... sample 12 ... Stage 13. Light source 14 ... Illumination optical system 15… Condenser lens 16 Objective lens 17 Projection lens 18 ... Shutter 19 ... Shutter driver 20 ... Defective pixel detection circuit 51 ... RAM 52 ... Defect address detection unit 53 ... Defect compensation circuit 531 ... Output selection unit 532, 534, 536 ... Adder 533, 535, 537 ... Multipliers 54 ... Address conversion unit 100 ... imaging device 200 ... Microscope

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 5/335 H04N 1/40 101G F-term (Reference) 5B047 AB04 BB04 BC05 BC07 BC11 BC14 CA04 CB09 CB22 DA06 5C024 CX26 CY35 EX05 HX14 HX50 HX57 5C065 BB17 BB23 DD17 EE05 EE06 GG13 5C077 LL20 MM03 MP08 PP23 PP32 PP37 PP46 PQ08 PQ18 SS04 TT09 5L096 AA02 AA06 BA20 CA14 DA01 FA15 GA12 LA05

Claims (8)

[Claims] An image pickup device for picking up a subject image; a displacement unit for periodically displacing the image pickup device and outputting an image signal of the subject image from the image pickup device for each displacement position; Image signal of the image sensor for each displacement position is input, and image processing means for rearranging the image signal and synthesizing an image, and among image data synthesized by the image processing means, An image pickup apparatus comprising: a defective pixel correction unit that performs different correction processing on image data synthesized based on an image signal from a defective pixel in accordance with a displacement position of the displacement unit. 2. The method according to claim 1, wherein the displacement means is 3 × at a 2/3 pixel interval.
The position of the image sensor is displaced to three positions to shift the pixels of the image signal output from the image sensor, and the defective pixel correction unit is different depending on the pixel shift position of the image signal output from the image sensor 2. The imaging apparatus according to claim 1, wherein a defective pixel correction value is obtained based on an image signal at an arbitrary position. 3. The defective pixel correcting means according to claim 1, wherein the defective pixel positioned in the left and right diagonally up and down direction with respect to the center pixel among the pixel shift positions of the image signal output from the image pickup device has four colors of the same color. A correction value is obtained by multiplying the sum of the signal levels of the pixels by 求 め, and adding the signal levels of the four pixels of the same color in the upper, lower, left, and right directions to the middle pixel for the defective pixel located in the up, down, left, and right directions. And a correction value obtained by multiplying the sum of the signal levels of two pixels of the same color, which are four pixels apart from each other at the left and right with respect to the defective pixel located in the middle, by ２, and the correction values correspond to the correction values. 3. The image pickup apparatus according to claim 2, wherein a correction process of the defective image is performed. 4. The image pickup device has a color filter for acquiring color discrimination information, and the defective pixel correction unit uses the color filter for a defective pixel of an image synthesized by the image processing unit. 2. The imaging apparatus according to claim 1, wherein different correction processes are performed according to the acquired color determination information and a displacement position by the displacement unit. 5. When the defective pixel is determined to be G (green) based on the color discrimination information, the defective pixel correcting means adds 1 to the sum of the signal levels of four pixels of the same color in the upper, lower, left, and right colors with respect to the defective pixel. 5. The imaging apparatus according to claim 4, wherein a correction value multiplied by / 4 is obtained, and a correction process is performed using the correction value. 6. The defective pixel correction unit further includes a position information conversion unit, which converts the position information of the image signal after the pixel shift by the position information conversion unit and divides the image signal into a plurality of image signals. 2. The imaging apparatus according to claim 1, wherein the processed image signals are sequentially processed. 7. A microscope for projecting a subject image onto the image sensor, a light blocking unit for selecting whether or not to project the subject image onto the image sensor, A defective pixel detecting unit that detects a defect of each pixel of an image signal output from the imaging element in a state where the projection is shielded, and a defect that performs a correction process on the defective pixel detected by the defective pixel detecting unit. 2. The imaging device according to claim 1, further comprising a pixel correction unit. 8. An image sensor for picking up a subject image is periodically displaced by a displacement means, and the image sensor outputs an image signal of the subject image for each displacement position, and an image signal for each displacement position. And an image combining method for combining images by rearranging the image data, wherein a displacement position of the image data to be combined based on an image signal from a defective pixel of the image sensor among the combined image data by the displacement unit An imaging method characterized in that different correction processes are performed in accordance with the conditions.