JP4481073B2 - Image correction apparatus, image correction method, and program - Google Patents

Description

  The present invention relates to an image processing technique, and more particularly to a technique for processing an image captured by a high-sensitivity solid-state imaging device suitable for a microscope camera or the like.

  2. Description of the Related Art Fluorescent microscopes are widely known in which proteins and genes are fluorescently labeled on biological tissues and cells so that proteins and genes can be observed and photographed with the naked eye. In particular, in recent years, it has been required to perform numerical analysis by photographing a substance that emits fluorescence only weakly with an electronic camera. For this purpose, a more sensitive electronic camera is required.

The electronic camera obtains a captured image signal by subjecting a subject image to photoelectric conversion by a solid-state imaging device such as a CCD (Charge Coupled Device).
FIG. 17 is a plan view showing an example of the element structure of the CCD image sensor.
In the CCD image pickup device shown in the figure, photodiodes 201 are arranged as a light receiving element in a matrix. A plurality of vertical transfer paths 202 are arranged in the column direction between the photodiodes 201, and one horizontal transfer path 203 is arranged in the row direction at the end of the vertical transfer path 202.

The signal charge accumulated in the photo diode 201 is read out to the vertical transfer path 202 in accordance with the charge transfer pulse TG, and transferred in the vertical direction in the vertical transfer path 202 in the figure.
The signal charge transferred through the vertical transfer path 202 is transferred to the horizontal transfer path 203, and further transferred to the left in the horizontal transfer path 203, and then finally passed through the charge detector 204. Is output by the read amplifier 205.

The vertical transfer path 202 transfers charges once every horizontal period read by the horizontal transfer path 203 in normal transfer, but discharges the charges in the vertical transfer path 202 by performing continuous operation during high-speed discharge. .
The cross-sectional view shown in FIG. 18 shows the structure of an interline CCD having a vertical overflow drain structure as an example of the element structure of the CCD image sensor.

  In the figure, an n-type semiconductor substrate 300 is formed by a first region 301 having a shallow junction and a second region 302 having a deep junction. Among these, the region where the junction n-type region of the first region 301 is formed functions as a photodiode, that is, a photoelectric conversion region (charge storage unit) 303.

  In the second region 302, a vertical shift register composed of a logic channel 304, that is, a transfer electrode 305 is formed, and the transfer electrode 305 is disposed on the main surface through an insulating layer 306. The logic channel 304 and the photoelectric conversion region 303 are separated by a channel stop region 307 made of a high p-type impurity layer, and a transfer gate (TG) region 308 is provided between the photoelectric conversion region 303 and the corresponding buried channel 304. Is arranged. Further, regions other than the photoelectric conversion region 303 are shielded from light by the metal layer 309.

  A substrate bias voltage VSUB311 which is a reverse bias voltage is applied to the junction between the N-type semiconductor substrate 300 and the first region 301 and the second region 302 which are P wells, and the first region of the P well immediately below the photoelectric conversion region 303 is applied. Blooming is suppressed by completely depleting 301 (depletion layer). Blue mink means that when intense light is incident on a pixel, the pixel is saturated and the signal charge overflows, so that it enters an adjacent pixel, signal line, vertical transfer path, etc. It is a phenomenon in which white parts like flowers bloom.

In order to improve the sensitivity of the CCD image pickup device, improvement of the photoelectric conversion efficiency of the photodiode is the most effective means.
Light incident on the photodiode is photoelectrically converted by the photodiode to generate a signal charge. Here, since the short wavelength light is largely attenuated in the photodiode, it is photoelectrically converted at a relatively shallow portion from the light receiving surface (semiconductor substrate surface). On the other hand, since long wavelength light has little attenuation, it reaches a relatively deep portion from the light receiving surface of the photodiode, and photoelectric conversion is also performed in the deep portion.

FIG. 19 is a graph showing the potential change in the depth direction from the light receiving surface of the photodiode. In the figure, the vertical axis represents the potential, and the horizontal axis represents the depth from the surface of the semiconductor substrate.
As described above, photoelectric conversion is performed at various depths of the semiconductor substrate. As shown in FIG. 19A, signal charges generated at a portion shallower than the overflow barrier OFB are accumulated in the photodiode, and accumulated. The signal charges thus transferred are transferred to the vertical transfer path and transferred. On the other hand, the signal charge generated in a portion deeper than the overflow barrier OFB is not accumulated in the photodiode but is swept out to the n-type semiconductor substrate 300.

Therefore, if the overflow barrier OFB is formed at a deeper position, the use of the signal charge generated in the deep part of the photodiode is promoted, so that the photoelectric conversion efficiency is improved as a whole, and as a result, the CCD imaging The sensitivity of the device is improved.
In order to deepen the overflow barrier OFB, there is a method of reducing the substrate bias voltage VSUB described above. When the substrate bias voltage VSUB is lowered, the potential curve changes, for example, from that shown in FIG. 19A to that shown in FIG. 19B, and the overflow barrier OFB is formed at a deeper position in the semiconductor substrate. As a result, the sensitivity of the CCD imaging device is improved.

  By the way, as disclosed in, for example, Patent Document 1, a recent CCD image pickup device includes a semiconductor substrate inside the CCD image pickup device in order to smoothly perform photoelectric conversion, signal charge accumulation, transfer, detection, and the like. A bias voltage (hereinafter also referred to as “substrate internal voltage”) is supplied.

FIG. 20 shows a configuration example of this type of solid-state imaging device and substrate bias generating circuit.
The substrate bias generation circuit 200 has resistors R1 and R2 for voltage division. One end of the resistor R1 is connected to the power supply line VCC, and the other end is connected to one end of the resistor R2. The other end of the resistor R2 is connected to the ground line GND.

  The base of the transistor TR1 is connected to the series connection point of the resistors R1 and R2, and the internal VSUB voltage generation circuit 206 is constituted by these. The substrate internal voltage VSUB is drawn from the emitter of the transistor TR1 and supplied to the semiconductor substrate on which the photodiode, the vertical transfer path, the horizontal transfer path 203, the charge detection unit 204, and the like described above are arranged.

  A substrate bias terminal CSUB is also provided at the series connection point of the voltage dividing resistors R1 and R2. The collector of the transistor TR2 is connected to the substrate bias terminal CSUB via an external resistor R3. The emitter of the transistor TR2 is connected to the ground line GND, and a terminal VSUBCont is provided at the base via a current limiting resistor R4.

  Here, when the potential of the terminal VSUBCont is set to H and the transistor TR2 is turned on, a part of the current that has been flowing through the resistor R2 flows through the resistor R3, and as a result, the potential of the substrate bias terminal CSUB decreases. . Then, since the substrate internal voltage VSUB is also lowered, the overflow barrier OFB is formed at a deeper position of the semiconductor substrate as described above, and the sensitivity of the CCD image pickup device is improved.

In addition, in a conventional electronic camera, position information of defects such as white spot scratches caused by various factors during the manufacturing process of the CCD is stored in a storage unit such as a ROM arranged in the camera, and the position information is stored. The position of the defect can be confirmed by referring to as necessary. Then, with respect to the fixed pattern noise (hereinafter, also referred to as “white spot noise” or “pixel defect” or the like) caused by such defects such as white spot scratches, as shown in FIG. Image interpolation processing for correcting pixel defects by replacing signals C1 and C3 of normal pixels in the vertical direction adjacent to signal B of the pixel to be generated (hereinafter referred to as defective pixels) or signals C2 and C4 of normal pixels in the horizontal direction Is generally applied (see, for example, Patent Document 2).

  It is known that this pixel defect includes a lot of white point noise that depends on exposure time (exposure time) due to dark current (current that flows due to application of a bias voltage for a semiconductor substrate) and temperature. ing. As illustrated in FIG. 22A, the output signal of white spot noise tends to increase as the exposure time increases. Further, as illustrated in FIG. 22B, the output signal of white spot noise tends to increase as the temperature of the CCD increases.

In addition, it is known that pixel defects may newly occur due to the influence of cosmic rays when a camera is transported by air in the Arctic Circle or the Antarctic Circle. For example, Patent Document 3 discloses such a case. In order to correct pixel defects that occur later, a technique for inspecting and correcting newly generated pixel defects in an image sensor has been disclosed.
JP 2000-22126 A JP 2003-348607 A JP 2002-125154 A

  As described above, when the substrate internal voltage VSUB is lowered, the overflow barrier OFB is formed at a deeper position of the semiconductor substrate, and the photoelectric conversion efficiency of the photodiode is improved. However, as can be seen with reference to FIG. 19B, when the substrate internal voltage VSUB is lowered, the potential increases, so that the signal charge storage capacity in the photodiode increases. Here, assuming that the subject is bright, a large amount of signal charge is accumulated in the photodiode, so that the surplus charge cannot be swept away to the semiconductor substrate, and as a result, the above-described blooming is likely to occur.

  Since photographing with a fluorescence microscope is intended for weak light, such blooming phenomenon does not become a problem in many cases. However, if there is white point noise that depends on the exposure time and temperature due to dark current, blooming phenomenon occurs due to the increase in white point noise, and interpolation based on the conventional four adjacent pixels is performed. The technique may not be able to correct it.

Here, as apparent from FIG. 22B, the output signal of white spot noise tends to decrease as the temperature of the CCD decreases. If white point noise is reduced, the blooming phenomenon can be prevented.
In order to lower the temperature of the CCD, cooling the CCD image pickup device is an effective means. However, adding a cooling element such as a Beltier element to the camera for cooling increases the cost.

  Further, in order to prevent white spot noise from causing the blooming phenomenon completely, it is necessary to cool down to about −30 ° C. in consideration of using a longer exposure time. The cooling requires a sealed structure for preventing condensation, which makes the camera more expensive.

  The present invention has been made in view of the above-described problems, and the problem to be solved is the influence of pixel defects of the image sensor without strongly cooling the image sensor even in high-sensitivity shooting. It is to obtain an appropriate image with suppressed.

An image correction apparatus according to an aspect of the present invention is based on the shape of pixels adjacent to pixels having pixel defects, which are caused by blooming phenomenon of the image sensor, among pixels constituting an image captured by the image sensor. A correction method determining means for determining a correction method for the pixel defect using a pixel in which a pixel defect is not generated among the pixels adjacent to the pixel in which the image has occurred, and the image pickup device according to the correction method according to the determination Correction means for correcting pixel defects of the image sensor on the image, and this feature solves the above-described problems.

The image correction apparatus according to the present invention described above further includes storage means for storing defect information that is information related to pixel defects of the image sensor, and the correction method determination means has a pixel defect from the defect information. You may make it acquire the position of the said pixel .

The image correction apparatus according to the present invention described above may further include a defect detection unit that detects pixel defects of the image sensor and generates the defect information.
In the above-described image correction apparatus according to the present invention, the determination by the correction method determination means is selection of a pixel that obtains a luminance value as a basis for calculating a luminance interpolation value of a pixel having a pixel defect. There may be.

  In the above-described image correction apparatus according to the present invention, the defect information, which is information related to the pixel defect of the image sensor, is at least one of the operating temperature and the exposure time of the image sensor when the image is captured by the image sensor. Further, defect information generating means for generating based on one side may be provided, and the correction method determining means may acquire the position of the pixel having a pixel defect from the defect information.

At this time, when the position of the pixel having the pixel defect acquired from the defect information is overlapped in the separate pixel, the correction method determining unit eliminates the overlap and then corrects the shape. The correction method may be determined based on the above.
In the above-described image correction apparatus according to the present invention, the imaging element captures a color image using a Bayer array, and the correction determination unit includes pixels included in an image captured by the imaging element. A correction method for the pixel defect may be determined based on a shape in which a plurality of pixels in which a pixel defect occurs and components of the same color are formed adjacent to each other.

Further, in the image correction apparatus according to the present invention described above, the determination by the correction method determination means is performed by selecting pixels of the same color component for obtaining a luminance value as a basis for calculating a luminance interpolation value of a pixel having a pixel defect. You may make it be.
The image correction apparatus according to the present invention described above further includes defect detection means for detecting a pixel defect of the image sensor from a dark signal of the image sensor and generating defect information that is information related to the pixel defect. The correction method determining means may acquire the position of the pixel having a pixel defect from the defect information.

At this time, the dark signal is a signal acquired to perform subtraction processing from the bright signal of the image sensor in noise reduction that reduces noise included in the image captured by the image sensor. May be.
An image correction method according to another aspect of the present invention is based on the shape of adjacent pixels having pixel defects, which are caused by blooming phenomenon of an image sensor, among pixels constituting an image captured by the image sensor. A correction method for the pixel defect using a pixel in which a pixel defect does not occur among pixels adjacent to the pixel in which the pixel defect has occurred is determined, and an image captured by the imaging element is determined according to the correction method according to the determination. Thus, the pixel defect of the image sensor is corrected, and the above-described problems are solved by this feature.

Of the pixels constituting the image picked up by the image sensor, the pixel among the pixels adjacent to the pixel in which the pixel defect is generated based on the shape of the pixel having a pixel defect adjacent to the pixel caused by the blooming phenomenon of the image sensor. A process for determining a correction method for the pixel defect using a pixel in which no defect has occurred, and correction of a pixel defect of the image sensor for an image captured by the image sensor in accordance with the correction method for the determination A program for causing a computer to perform processing is also according to the present invention, and the above-described problems are solved by causing the computer to execute the program.

  The present invention has an effect that an appropriate image can be obtained without strongly cooling the image sensor even when photographing with high sensitivity.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 shows the configuration of a microscope system that implements the present invention. This microscope system mainly shows a configuration for performing fluorescence observation.
The microscope system shown in FIG. 1 includes an epi-illumination optical system 12 as an illumination system. The epi-illumination optical system 12 includes an epi-illumination light source 21. A dichroic mirror 32 is disposed at a position where the epi-illumination optical system 12 and the observation optical path S intersect.

  On the observation optical path S, a revolver 28 for selecting one of a sample stage 26 on which a specimen to be observed is mounted and a plurality of mounted objective lenses 27 by rotating operation to be positioned on the observation optical path S. A cube unit 30 for switching the dichroic mirror 32 on the observation optical path S and a beam splitter 31 for branching the observation optical path S into an observation optical path S ′ and an observation optical path S ″ are arranged. Are arranged in the trinocular tube unit 5.

  On the observation optical path S ′ bent forward by the beam splitter 31, an eyepiece unit 6 having an eyepiece 6 a is disposed. On the other hand, the imaging lens unit 100 and the electronic camera 36 are installed and arranged on the microscope main body 1 on the observation optical path S ″ transmitted through the beam splitter 31. An imaging element 42 is arranged in the electronic camera 36. The optical image from the objective lens 27 is imaged on the imaging surface of the imaging element 42 by the imaging lens 100 a in the imaging lens unit 100.

The revolver 28, the cube unit 30, and the beam splitter 31 are each electrically driven, and are driven by respective motors (not shown) according to the respective drive signals from the microscope drive circuit 37.
The microscope control unit 41 controls the operation of the entire microscope system, and is connected to the epi-illumination light source 21, the microscope drive circuit 37, and the electronic camera 36. The microscope control unit 41 performs dimming of the epi-illumination light source 21 and gives a control instruction to the microscope drive circuit 37 according to a CPU 60 (described later) that determines imaging conditions and the like with the electronic camera 36. Further, the microscope control unit 41 feeds back the control state for the epi-illumination light source 21 and the control state for the microscope drive circuit 37 to the electronic camera 36.

  FIG. 2 is a block diagram showing a configuration of the electronic camera 36 used in the microscope system shown in FIG. The electronic camera 36 includes a CPU (Central Processing Unit) 60 that controls the microscope control unit 41. The image sensor 42 provided in the electronic camera 36 captures a color image or a monochrome image, and is disposed on the observation optical path S ″.

A solid-state imaging device (hereinafter simply referred to as “CCD”) 42 such as a CCD captures and photoelectrically converts an observation image of the specimen 3 magnified by the microscope system shown in FIG.
A CDS (Correlated Double Sampling) circuit 43 extracts an image signal component from the output signal of the CCD 42.

The amplifier (AMP) 44 is a signal level control means including an AGC (Automatic Gain Control) circuit for adjusting the output signal level of the CDS circuit 43 to a predetermined value.
The A / D converter 45 converts the analog signal output from the AMP 44 into a digital signal. In the present embodiment, the luminance value of each pixel in the image captured by the CCD 42 is expressed by 4096 gradations (12 bits).

The image memory 46 stores the digital signal output from the A / D converter 45.
The memory controller 55 controls the image memory 46.
The light shielding leak detection circuit 52 confirms the light shielding state of the CCD 42 based on the image signal read from the image memory 46.

The defect detection circuit 62 detects a defective pixel existing in the CCD 42.
The defect memory 63 includes an EEPROM (Electrically Erasable and Programmable Read only Memory) or the like, and defect information such as an address of a defective pixel existing in the CCD 42 is stored in advance.

The defect correction circuit 64 corrects the image signal affected by the defective pixel existing in the CCD 42.
The image signal processing circuit 51 performs image processing such as γ correction and edge enhancement on the image signal corrected by the defect correction circuit 64.

A liquid crystal display (LCD) 59 is a display unit that includes a signal processing circuit that processes an image signal into a displayable form and displays an image represented by the image signal.
A DRAM (Dynamic Random Access memory) 56 is a camera built-in storage means including a memory or the like for temporarily storing image signals.

The compression / decompression circuit 57 performs compression processing and expansion processing on the image signal.
The recording medium 58 is a memory card, for example, and stores an image signal.
The operation unit 61 is a plurality of switches including an AF (Auto Focus) operation at the time of shooting and a trigger switch that can generate a trigger signal for starting an exposure operation, a rotation operation switch of the cube unit 30, and the like. is there.

The temperature sensor 47 is temperature detection means for detecting the temperature state of the CCD 42.
A timing generator (TG) 53 generates a synchronization signal such as a drive pulse for the CCD 42.
The signal generator (SG) 54 supplies a synchronization signal to the TG 53.

The SUB pulse superimposing circuit 49 is a circuit for inputting the SUB pulse clamped to the internal voltage of the CCD 42 to the CCD 42, for example, the circuit shown in FIG.
The CCD SUB voltage switching circuit 48 is a circuit for switching the substrate internal voltage VSUB, for example, the circuit shown in FIG.

  Each component of the electronic camera 36 described above is electrically connected to a CPU 60 that is a control means, and the entire microscope system shown in FIG. Here, in particular, the CCD 42 has an electronic shutter function (means) that is controlled by an output from the SUB voltage superimposing circuit 49 and the like, so that the exposure time can be controlled.

Next, the operation of the microscope system having such a configuration will be described. Of the operations performed at the time of shooting, only the portion related to the present invention will be described.
As described above, in the microscope system in FIG. 1, the CPU 60 performs all the control in an integrated manner.

  In the fluorescence observation method, the light emitted from the epi-illumination optical system 12 is dispersed by the fluorescent cube in the cube unit 30 and is reduced through the objective lens 27 to irradiate the specimen 3. The specimen 3 generates weak fluorescence by the irradiated light, and this fluorescence is magnified through the objective lens 27 and dispersed by the fluorescent cube in the cube unit 30 at a wavelength different from the above-described spectrum and irradiates the CCD 42.

  The CCD 42 performs exposure (charge accumulation) and signal readout under drive control by the TG 53. An image signal component is extracted from the read signal by the CDS circuit 43, the output signal is adjusted to a predetermined level in the AMP 44, and converted into a digital signal by the A / D converter 45. The image signal converted into the digital signal is temporarily stored in the image memory 46.

The image signal stored in the image memory 46 is input to the defect correction circuit 64, and the portion corresponding to the defective pixel is corrected based on the stored contents of the defective memory 63 in which the address of the defective pixel existing in the CCD 42 is stored. .
The image signal in which the defective pixel is corrected by the defect correction circuit 64 is subjected to image processing such as γ correction and edge enhancement by the image signal processing circuit 51, and is then input to the LCD 59 to reproduce and display the fluorescent image of the sample 3. Is made.

The above is the operation of photographing and image display of the microscope system whose configuration is shown in FIGS.
The drive control of the CCD 42 is performed by various drive signals (vertical drive pulse, horizontal drive pulse, substrate internal voltage VSUB, etc.) output from the TG 53. The CCD 42 is, for example, an interline type vertical overflow drain (VOFD). ) An imaging device having a structure is used.

  A plan view and a cross-sectional structure of the CCD 42 are shown in FIGS. Here, as described above, the substrate internal voltage VSUB is a substrate bias voltage for determining the maximum charge accumulation level (overflow level: OFL) of the photodiode (PD) which is a photoelectric conversion element. When a value pulse (VSUB pulse) is superimposed, the charge of each PD can be discharged to the substrate and the accumulated charge can be reset.

  As shown in FIG. 19B, the overflow barrier OFB is formed in a deeper part than usual by making the value of the substrate internal voltage VSUB smaller than usual, and the sensitivity of the CCD 42 is improved. Therefore, in the following description, a case where the value of the substrate internal voltage VSUB is normal is referred to as a normal mode, and a case where the value of the substrate internal voltage VSUB is reduced is referred to as a high sensitivity mode. In the electronic camera 36, a normal mode in which the sensitivity of the CCD 42 is normal sensitivity (hereinafter referred to as “normal sensitivity”) and a high sensitivity mode in which sensitivity is improved can be set.

The configurations of the SUB voltage switching circuit 48 and the SUB pulse superimposing circuit 49 in the CCD 42 are the same as those shown in FIG.
Next, FIG. 3 will be described. This figure shows a first example of a detailed configuration of the defect correction circuit 64 in FIG.

  As described above, the defect detection circuit 62 detects a defective pixel existing in the CCD 42. This defective pixel is detected, for example, in the initial state, that is, before shipment from the factory that manufactured the microscope system shown in FIG. At this time, if the value of the substrate internal voltage VSUB is made smaller than normal, the sensitivity of the CCD 42 is improved as shown in FIG. 19B. On the other hand, when the substrate internal voltage VSUB is set to a normal value, an image is obtained. The dot-like pixel defects which have been discretely generated above are enlarged, and pixel defects are also generated in pixels located adjacent to the periphery thereof, resulting in the noise of the above-described blooming phenomenon.

  In the present embodiment, the defect detection circuit 62 is also made to detect defective pixels in a state where the value of the substrate internal voltage VSUB is smaller than usual and the sensitivity of the CCD 42 is set to a high sensitivity state. Information (defect position information) indicating the position of the defective pixel detected at this time is processed by the CPU 60, and defective pixels located adjacent to each other which become blooming noise are integrated.

  Then, information indicating the position of the pixel representing the integrated defective pixel group (center defective pixel) (center defective pixel position information) and information indicating the shape of the defective pixel group (defect shape information) are obtained to obtain a defective memory. 63 is stored in advance. Note that the center defective pixel at this time is a pixel in which a pixel defect occurs even when the value of the substrate internal voltage VSUB is set to a normal magnitude and the sensitivity of the CCD 42 is set to the normal state.

  The interpolation method calculation unit 73 reads out the above-described center defective pixel position information and defect shape information from the defect memory 63, obtains position information for each pixel forming the defective pixel group, and adjoins the defective pixels. An interpolation method for correcting each defective pixel is determined based on the state, and the obtained defect position information and information indicating the interpolation method are stored in the RAM 71.

The defect position detection unit 72 interpolates pixels at the position indicated by the defect position information stored in the RAM 71 among the pixels in the image indicated by the image data for the specimen 3 read from the image memory 46. The calculation unit 74 is notified.
The interpolation calculation unit 74 interpolates the data (luminance data) about the pixel notified from the defect position detection unit 72 using the interpolation method stored in the RAM 71.

Here, FIG. 4 will be described. This figure shows the structure of data stored in the defective memory 63.
As described above, the defect memory 63 stores center defect pixel position information and defect shape information. Among these, the central defect pixel position information is composed of a horizontal defect address indicating the horizontal position of the central defective pixel in the image and a vertical defect address indicating the vertical position of the central defective pixel in the image.

The defect shape information takes into account the defect shape in which the pixel defect due to the blooming phenomenon spreads radially from the central defect pixel in one or two dimensions, and changes the defect shape according to the extent of the defect pixel spread. Information indicating the rank when ranking is performed.
In the present embodiment, as shown in FIG. 5, three ranks are defined as defect shapes. In the same figure, (a) shows the defect shape of rank L1 in which pixel defects exist in both the top and bottom or left and right pixels adjacent to the central defective pixel, and (b) adjacent to the central defective pixel. The defect shape of rank L2 in which pixel defects exist in all four pixels in total, up and down and left and right, and (c) shows a total of eight pixels adjacent to the center defective pixel, up and down, left and right, and diagonal. The defect shape of rank L3 in which there are pixel defects in all of FIG.

  Considering the appropriateness of pixel interpolation, there is a limit to the interpolation of the central defective pixel based on the luminance value of the surrounding pixels, and if the limit is exceeded, useful correction is applied to the image. I can't. The threshold bit in the defect shape information of FIG. 4 is information indicating whether or not such useful correction can be performed, and whether or not the interpolation calculation unit 74 performs interpolation of the defective pixel based on such a threshold bit. You may make it control.

Next, a defective pixel interpolation method will be described with reference to FIG. This figure exemplifies a case where a defect shape of rank L2 in FIG. 5 exists.
In FIG. 6A, the defective pixels are A2, B1, B2, B3, and C2, and A1, A3, C1, and C3 are normal pixels that do not have a pixel defect. In this case, the luminance value of each defective pixel is calculated by the following formula based on the luminance value of the normal pixel.

A2 = (A1 + A3) / 2
B1 = (A1 + C1) / 2
B3 = (A3 + C3) / 2
C2 = (C1 + C3) / 2
B2 = (A1 + A3 + C1 + C3) / 4
That is, as shown in FIG. 6B, the interpolation luminance value of the defective pixel is calculated by averaging the luminance values of normal pixels located around the defective pixel to be interpolated.

  In the present embodiment, 18 patterns from K0 to K17 shown in FIG. 7 are defined in advance as the above-described addition average pattern. When calculating the luminance value of the defective pixel, one of these patterns is selected by the interpolation method calculation unit 73 based on the positional relationship between the defective pixel to be corrected and normal pixels located around the defective pixel. .

Note that a pattern K0 in FIG. 7 indicates that no interpolation is performed.
If the sensitivity of the CCD 42 is lowered to the normal sensitivity by controlling the VSUB voltage by the SUB voltage switching circuit 48, it is possible to prevent the adjacent state of defective pixels, which is a blooming phenomenon, from occurring. The pixel defect generated at this time is a point defect in which defective pixels are isolated (not adjacent) (a CCD 42 having an adjacent pixel defect despite being set to normal sensitivity is regarded as a defective product, and FIG. However, since defect information of the central defective pixel is stored in the defect memory 63, if this defect information is used, an image captured with the sensitivity of the CCD 42 as the normal sensitivity is used. However, the defective pixel can be corrected in the same manner.

Next, FIG. 8 will be described. This figure is a modification of the configuration of the defect correction circuit 64 shown in FIG.
In the configuration shown in FIG. 8, based on the defect position information detected by the defect detection circuit 62, integration of adjacent defective pixels and a pixel representing the integrated defective pixel group (central defect pixel) Acquisition of information indicating the position of the pixel (center defect pixel position information) and information indicating the shape of the defective pixel group (defect shape information) is performed by an electronic computer 80 such as a computer or a workstation instead of the CPU 60. It is to do. In this case, the obtained center defect pixel position information and defect shape information may be stored in an external recording device 81 such as an HDD (Hard Disk Drive).

  Further, in the configuration shown in FIG. 8, the CPU 60 or the electronic computer 80 determines the interpolation method of each defective pixel based on the adjacent state of the defective pixel instead of the interpolation method calculation unit 73. In this case, the imaging condition of the image indicated by the image data stored in the image memory 46, more specifically, the sensitivity setting of the CCD 42 cannot be known before determining the interpolation method. The interpolation method is determined in advance in each case where the sensitivity setting is a normal sensitivity setting and a high sensitivity setting, and the result is stored in the RAM 71. As a result, the configuration of the defect correction circuit 64 becomes simple because the interpolation method calculation unit 73 is deleted.

In addition to this, the CPU 60 can perform the entire process of pixel defect detection and correction of the CCD 42 performed by the defect detection circuit 62 and the defect correction circuit 64.
Here, FIG. 9 will be described. FIG. 6 is a flowchart showing the processing contents of the pixel defect detection and correction processing performed by the CPU 60. In order to cause the CPU 60 to perform the process shown in FIG. 8, a control program for causing the CPU 60 to perform this process is created and stored in a storage device (not shown) such as a ROM, and the control program is read into the CPU 60. And just execute it.

First, the process shown in FIG. 9A will be described. This process is a defect information acquisition process, which is a process for detecting a pixel defect of the CCD 42 and storing defect information about the detected pixel defect in the defect memory 63.
This process is started when, for example, a predetermined operation on the operation unit 61 indicating the start of execution of this process is detected in an initial state before factory shipment.

First, in S101, processing for setting shooting conditions, more specifically, sensitivity setting of the CCD 42, and setting exposure time and the like to predetermined settings for acquiring defective pixels is performed.
In S102, processing for capturing a light-shielded image is performed under the imaging conditions set by the processing in S101.

In S103, a process of extracting white point noise in the light-shielded image acquired by the process of S102, that is, a pixel having a pixel defect is performed.
In S104, processing for integrating adjacent defective pixels extracted is performed. At this time, the acquisition of the center defective pixel position information is performed.

In S105, the integrated defective pixel group is ranked into the ranks illustrated in FIG. 5, and defect shape information is acquired.
In S106, a process of storing the central defect pixel position information and defect shape information for each defective pixel obtained by the above processes as defect information in the defect memory 63 is performed, and thereafter, the defect information acquisition process is completed. .

  Next, the process shown in FIG. 9B will be described. This process is a defective pixel correction process, and is a process for correcting pixel defects for the image data of the specimen 3 stored in the image memory 46. This process is started when the storage of the image data of the specimen 3 in the image memory 46 is completed.

First, in S111, defect information is read from the defect memory 63, and a process of recognizing the state of pixel defects in the CCD 42 is performed.
In S112, attention is paid to one of the defective pixels to be corrected, setting of shooting conditions (sensitivity setting of the CCD 42) at the time of shooting of the correction target image whose image data is stored in the image memory 46, and the defect memory 63. 7 is determined based on the adjacent shape (rank) of the defective pixel with respect to the defective pixel to be corrected, which is indicated by the defect information read out from (the correction method illustrated in FIG. 7) Is selected).

In S113, a process of obtaining the luminance value of normal pixels around the defective pixel to be corrected from the image data of the image memory 46, which is necessary in the interpolation method determined by the process of S112, is performed.
In S114, according to the interpolation method determined by the process of S112, the process of calculating the interpolated brightness value for the defective pixel to be corrected is performed using the brightness value acquired by the process of S113.

In S115, the luminance value of the defective pixel to be corrected is replaced with the interpolated luminance value calculated in S114 and output to the image signal processing circuit 51.
In S116, a process is performed to determine whether or not an interpolation luminance value is not calculated among the defective pixels indicated by the defect information acquired in the process of S111. If the determination result is “Yes”, the process returns to S112, and the process of calculating the interpolated luminance value for the uncorrected defective pixel is repeated. On the other hand, if the calculation of the interpolated luminance values for all defective pixels indicated in the defect information has been completed, the defective pixel correction process is terminated.

Even if the CPU 60 performs the defect information acquisition process and the defective pixel correction process described above, the pixel defect of the CCD 42 can be corrected appropriately.
As described above, according to the above-described embodiment, when the sensitivity of the CCD 42 is set to a high sensitivity and the specimen 3 is imaged and a pixel defect due to the blooming phenomenon occurs, it is stored in the storage memory 63. Since the adjacent relationship between the defective pixels is grasped from the central defective pixel position information and the defect shape information, and the interpolated value of the luminance of the defective pixel is obtained from the average of the luminance values of the normal pixels located around each defective pixel, the pixel defect It is possible to provide a high-sensitivity image of the specimen 3 in which the influence of the above is suppressed. Further, an image in which the influence of the pixel defect is suppressed is provided even when the specimen 3 is imaged with the sensitivity of the CCD 42 set to the normal sensitivity, which does not occur in the pixel defect due to the blooming phenomenon. Can do.

Next, another embodiment of the present invention will be described. In order to distinguish the embodiment described heretofore from what has been described so far, the embodiment described so far will be referred to as the first embodiment, and the embodiment described heretofore will be referred to as the second embodiment. It shall be called.
In the first embodiment, for example, an initial state, that is, a center obtained based on the result of pixel defect detection of the CCD 42 performed by the defect detection circuit 62 before shipment from the factory that manufactured the microscope system shown in FIG. Defective pixel position information and defect shape information are stored in the defect memory 63, and pixel defects are corrected based on these information. However, as described with reference to FIG. 22, the defect shape of the pixel defect changes depending on the exposure time of the CCD 42 at the time of imaging and the operating temperature of the CCD 42. Therefore, in the second embodiment, the defect shape information is determined based on the exposure time of the CCD 42 at the time of imaging and the operating temperature of the CCD 42.

Since the configuration of the microscope system according to the second embodiment is the same as the configuration of the microscope system according to the first embodiment shown in FIGS. 1 and 2, the description thereof is omitted.
As already described with reference to FIG. 2, the CCD 42 has an electronic shutter function (means) controlled by the output from the SUB voltage superimposing circuit 49 and the like so that the CPU 60 can control the exposure time. . In addition, the temperature sensor 47 allows the CPU 60 to know the temperature state of the CCD 42.

Here, FIG. 10 will be described. This figure shows a third example of the detailed configuration of the defect correction circuit 64, and shows the configuration according to the second embodiment.
As described above, in the second embodiment, the defect memory 63 stores only the center defect pixel position information.

  The defect shape calculation unit 75 receives information indicating the exposure time at the time of imaging of the specimen 3 and temperature information of the CCD 42 from the CPU 60, and is indicated by central defect pixel position information stored in the defect memory 63 based on these information. The defect shape for the central defective pixel is derived. The method for deriving the defect shape is, for example, measuring the exposure time and temperature dependence of the dark white defect (relationship between the exposure time and temperature and the defect shape) for the CCD 42 before shipment from the factory. Derived using this measurement result.

If the manufacturing variation is sufficiently small, the same measurement result may be used without separately measuring the CCD 42.
Also, for example, a defect shape generated in the CCD 42 when a predetermined exposure time (preferably a short time) and temperature (preferably a low temperature) are set as the initial state is obtained, and the defect shape in this initial state is the same as described above. Then, the rank of the defect shape in the initial state is changed based on the amount of change from the initial state with respect to the exposure time and temperature of the CCD 42 when the sample 3 is imaged, and the sample 3 is imaged. The rank of the defect shape of the CCD 42 at the time may be obtained.

As described above, according to the second embodiment, since the defect shape of the pixel defect is derived based on the exposure time and temperature of the CCD 42 at the time of image capturing, more appropriate pixel defect correction is performed on the image. Is possible.
Next, a third embodiment will be described.

  In the third embodiment, the pixel defects in the image captured by the CCD 42 set to the normal sensitivity are isolated from each other. However, when the CCD 42 is set to the high sensitivity and imaged, the pixel defects are surrounded by the normal sensitivity. Correction of pixel defects is performed when pixel defects caused by the blooming phenomenon that overlap (hereinafter, this case is referred to as “when adjacent defects occur”).

The configuration of the microscope system according to the third embodiment is the same as the configuration of the microscope system according to the first embodiment shown in FIGS. The detailed configuration of the defect correction circuit 64 is also the same as that shown in FIG.
Here, FIG. 11 will be described. This figure is a flowchart showing the processing contents of the interpolation method determination processing when adjacent defects may occur.

The process shown in the figure is performed by the interpolation method calculation unit 73, and more specifically shows the process of S112 in the defective pixel correction process shown in FIG. 9B.
First, in S201, the defect information is read from the defect memory 63 to obtain the center defect pixel position information and the defect shape information for the CCD 42. In the subsequent S202, based on the acquired information, each defect pixel position information is obtained. Processing for checking the adjacent state is performed.

  Here, in S203, a process for determining whether or not the above-described adjacent defect, that is, an overlapping state of defective pixels has occurred is performed. Here, if an overlapping state has occurred, one of the information on the defective pixel that overlaps is deleted to eliminate the overlapping state, and the process returns to S202 to check the adjacent state of each defective pixel. Repeat the process. On the other hand, if there is no overlapping state between the defective pixels, the process proceeds to S204, and the interpolation method of the defective pixel to be corrected is obtained from the information on the center defective pixel position and the defect shape in the state where the overlapping state is eliminated. Is determined (selecting one of the correction methods illustrated in FIG. 7), and the interpolation method determination process is terminated.

The method for determining the interpolation method for the defective pixel to be corrected, which is performed as the processing of S204 in the above-described interpolation method determination processing, will be described with reference to the example of FIG.
The example of FIG. 12 shows a case where a pixel defect of rank L2 has occurred, where the B3 and C2 pixels in the image region composed of the pixels of horizontal 4 rows × vertical 4 columns are the central defective pixels, respectively. As shown in FIG. 12A, overlap occurs in the defective pixels B2 and C3, resulting in adjacent defects.

  Here, considering B2 as the center, since pixels A3, B3, C1, C2, and C3 among the pixels adjacent to B2 are defective pixels, the luminance values of these pixels are used to calculate the interpolated luminance value of B2. Do not use. That is, in this case, the interpolated luminance value of the defective pixel B2 is calculated by the following equation using the luminance values of the adjacent normal pixels A2 and B1.

B2 = (A2 + B1) / 2
That is, for B2, the interpolated luminance value of B2 is calculated by the averaging using the pattern K8 in FIG.
Hereinafter, similarly, the interpolated luminance value of each defective pixel is calculated by the following equation, and therefore the following pattern in FIG. 7 is selected.

A3 = (A2 + A4) / 2 ......... K2
B3 = (A4 + C4) / 2... K13
B4 = (A4 + C4) / 2... K1
C1 = (B1 + D1) / 2... K1
C2 = (B1 + D1) / 2... K12
C3 = (C4 + D3) / 2... K11
D2 = (D1 + D3) / 2 ......... K2
FIG. 12B shows the relationship between the defective pixel having the defect shape shown in FIG. 12A and the normal pixel used to calculate the interpolated luminance value of each defective pixel based on the above formula. This is indicated by an arrow.

Note that, when the defect shape due to the adjacent defect spreads greatly and interpolation with the pattern shown in FIG. 7 cannot be performed, the interpolation is not performed.
As described above, according to the third embodiment, pixel defects are appropriately corrected even for an image in which adjacent defects are generated.

Next, a fourth embodiment will be described.
In the fourth embodiment, pixel defects are corrected when the CCD 42 is a Bayer type single-plate color CCD.
In a Bayer array single-chip color CCD, the array of light receiving elements differs depending on the color to be detected for luminance, and 50% of the pixels per unit area occupy the green component (G), and the remaining 25%. Are occupied by pixels of the red component (R) and the blue component (B), respectively. In addition, the same color pixels are located in four pixels on the upper, lower, left, and right sides of one pixel of the green component (G) for the red component (R) and the blue component (B). The pixel (G) is located at four pixels adjacent in the oblique direction.

In the present application, for convenience of explanation, among the positional relationships of the same color component pixels with respect to the red component (R) and blue component (B) pixels, the above, below, left, right, and right sides sandwiching one pixel of the green component (G). The positional relationship of the four pixels is referred to as “adjacent”.
A method for interpolating defective pixels in color pixels will be described with reference to FIG.

  The pixel defect illustrated in FIG. 13A indicates a case where the central defective pixel is a red component (R) pixel and a pixel defect of rank L2 occurs, and the pixel illustrated in FIG. The defect indicates a case where the central defective pixel is a green component (G) pixel and a pixel defect of rank L2 occurs. As can be seen from FIG. 13, in any of the cases shown in FIG. 13, the interpolated luminance value for the defective pixel is the defect of the interpolation target among the normal pixels adjacent to the defective pixel of the interpolation target. Calculation is made using the luminance value of the same color as the pixel.

The configuration of the microscope system according to the fourth embodiment is the same as the configuration of the microscope system according to the first embodiment shown in FIGS. 1 and 2. The detailed configuration of the defect correction circuit 64 is also the same as that shown in FIG.
Here, FIG. 14 will be described. This figure is a flowchart showing the contents of the interpolation method determination process when a color pixel is targeted.

The process shown in the figure is performed by the interpolation method calculation unit 73, and more specifically shows the process of S112 in the defective pixel correction process shown in FIG. 9B.
First, in S301, the defect information is read from the defect memory 63 to obtain the center defect pixel position information and the defect shape information for the CCD 42, and the color component of the center defect pixel associated with the center defect pixel position information is obtained. Information is acquired, and in the subsequent S302, based on the acquired information, a process for examining the adjacent state of each defective pixel for each color component is performed.

  Here, in S303, a process for determining whether or not the above-described adjacent defect, that is, an overlapping state of defective pixels has occurred is performed. Here, if an overlapping state has occurred, one of the information about the defective pixels that overlap is deleted to eliminate the overlapping state, and the process returns to S302 to change the adjacent state of each defective pixel to a color. Repeat the process for each component. On the other hand, if the overlap state between the defective pixels does not occur, the process proceeds to S304, and the correction target in the case of the color pixel is determined from the information on the center defect pixel position and the defect shape in the state where the overlap state is eliminated. Processing for determining an interpolation method for defective pixels is performed, and the interpolation method determination processing ends.

A method for determining an interpolation method for a defective pixel to be corrected in the case of a color pixel, which is performed as the processing of S304 in the above-described interpolation method determination processing, will be described with reference to an example of FIG.
The example of FIG. 15 is a pixel of rank L2 in which G8 and G11 pixels (both green component pixels) in the image area composed of color pixels having a Bayer array of 6 rows x 6 columns are center defective pixels, respectively. FIG. 15A shows a case where a defect has occurred. Here, as shown in FIG. 15A, overlap occurs in the defective pixel R5 (red component pixel) and B5 (blue component pixel), and the adjacent defect. Has occurred.

  Here, considering R5 as the center, among the pixels of the same color adjacent to R5, since R6 and R8 are defective pixels, the luminance values of these pixels are not used for calculating the interpolated luminance value of R5. To. In other words, in this case, the interpolation luminance value of the defective pixel R5 is determined by using the luminance values of the normal pixels R2 and R4 of the same color adjacent to R5 and calculated by the following formula.

R5 = (R2 + R4) / 2
Hereinafter, similarly, the interpolation method is determined so that the interpolation luminance value of each defective pixel is calculated by the following equation.
B2 = (B1 + B3) / 2
G8 = (G6 + G12) / 2
R6 = (R3 + R9) / 2
B4 = (B1 + B7) / 2
G11 = (G7 + G13) / 2
B5 = (B6 + B8) / 2
R8 = (R7 + R9) / 2
FIG. 15B shows the relationship between the defective pixel having the defect shape shown in FIG. 15A and the normal pixel used to calculate the interpolated luminance value of each defective pixel based on the above equation. This is indicated by an arrow. As is clear from the figure, the interpolation luminance value for the defective pixel is the luminance value for the normal pixel located around the defective pixel that is the interpolation target and the same color as the defective pixel that is the interpolation target. Is calculated using

As described above, according to the fourth embodiment, when the CCD 42 is a Bayer type single-plate color CCD, pixel defects with respect to a color image in which adjacent defects occur are appropriately corrected.
Next, a fifth embodiment will be described.
In general, a random dark white defect signal whose generation amount depends on the exposure time and temperature appears prominently in the output signal of the light-shielded CCD. Therefore, the S / N (signal-to-noise ratio) of the captured image can be improved by subtracting the CCD output signal (dark signal) in the light-shielded state from the CCD output signal (light signal) during image capturing. it can. Such subtraction processing is called noise reduction. However, when fixed pattern noise due to pixel defects as described above occurs in the dark signal, the quality of the captured image is degraded. Therefore, when noise reduction is performed, it is necessary to correct defective pixels even for dark signals.

The fifth embodiment provides correction of pixel defects when performing noise reduction.
The configuration of the microscope system according to the fourth embodiment is the same as the configuration of the microscope system according to the first embodiment shown in FIGS. 1 and 2 except for the portion shown in FIG.
FIG. 16 will be described. This figure shows a fourth example of the detailed configuration of the defect correction circuit, and shows a configuration example in the case where the noise reduction circuit is arranged in front of the defect correction circuit 64.

Of the image data output from the A / D converter 45, the image data of the sample 3 (image data of the light signal) is stored in the image memory A 46a, and the image data when the CCD 42 is in a light-shielded state (dark). The image data of the time signal) is stored in the image memory B46b.
The subtraction circuit 65 performs a noise reduction process for subtracting the dark signal image data read from the image memory B 46b from the bright signal image data read from the image memory A 46a.

The selector 67 selects one of the image signal of the bright signal read from the image memory 46 a and the image subjected to the noise reduction process output from the subtraction circuit 65.
The defect detection circuit 62 detects a defective pixel existing in the CCD 42 from the image data of the dark signal read from the image memory 46b.

The defect memory 63 is composed of an EEPROM or the like, and stores the central defect pixel position information and defect shape information about the CCD 42 in advance as in the first embodiment.
The defect correction circuit 64 corrects an image signal affected by a defective pixel existing in the CCD 42, and includes the following components.

  The interpolation method calculation unit 76 uses pixel position information and defect shape information about the defective pixel in the CCD 42 detected by the defect detection circuit 62 from the image data of the dark signal for each pixel forming the defective pixel group. 3 is determined, and an interpolation method for correcting each defective pixel is determined based on the adjacent state of the defective pixels. The operation itself is the interpolation method in the first embodiment shown in FIG. The calculation unit 73 is the same.

  The defect position detection unit 77 is arranged at the position of each defective pixel obtained by the interpolation method calculation unit 76 among the pixels in the image indicated by the image data for the sample 3 read from the image memory 46b. The operation itself is the same as that of the defect position detector 72 in the first embodiment shown in FIG.

The interpolation method calculation unit 73, the RAM 71, the defect position detection unit 72, and the interpolation calculation unit 74 are all the same as those in the first embodiment shown in FIG.
Whether the priority selection unit 78 assumes that the interpolation method to be performed by the interpolation calculation unit 74 is indicated by the information stored in the RAM 71 or is determined by the interpolation method calculation unit 76 The selection is made, that is, the interpolation method based on the pixel defect information stored in the defect memory 63 or the interpolation method based on the image data of the dark signal obtained most recently from the CCD 42 is selected. Do.

The defect correction circuit 64 shown in FIG. 16 is configured as described above.
Next, the operation of the defect correction circuit 64 shown in FIG. 16 will be described.
The subtraction process in the noise reduction is performed by alternately switching the state of the CCD 42 between the imaging state (non-light-shielding state) and the light-shielding state of the specimen 3 when the specimen 3 is imaged. Here, in order to display a captured image in real time, it is necessary to correct pixel defects in real time.

  When the noise reduction is not functioned, the selector 67 is switched so that the image data of the light signal stored in the image memory 46a is input to the defect correction circuit 64, and the notification (defect position detection unit 72) The priority selection unit 78 is switched so that the interpolation method (interpolation method A) stored in the RAM 71 is input to the interpolation calculation unit 74. At this time, the operation of the defect correction circuit 64 is the same as that in the first embodiment shown in FIG.

  On the other hand, in the case of making noise reduction function, the selector 67 is switched so that the image data of the dark signal stored in the image memory 46 b is input to the defect correction circuit 64, and the detection result ( The priority selection unit 78 is switched so that the defect position B) and the interpolation method (interpolation method B) determined by the interpolation method calculation unit 76 are input to the interpolation calculation unit 74. That is, instead of correcting the pixel defect based on the pixel defect of the CCD 42 detected in the initial state before shipment from the factory, the darkness of the CCD 42 in the light-shielded state acquired for noise reduction when the sample 3 is imaged. The pixel defect is corrected based on the pixel defect of the CCD 42 detected from the time signal. By doing so, correction of pixel defects is appropriately performed when noise reduction is performed.

  Note that when noise reduction functions, priority is given to an operation in which both the combination of the defect position A and the interpolation method A described above and the combination of the defect position B and the interpolation method B are input to the interpolation calculation unit 74. You may make it make the part 78 perform. At this time, if the defect position A and the defect position B overlap, any one of the combinations is input to the interpolation calculation unit 74. In this case, it is preferable that a combination of the defect position A and the interpolation method A is input to the interpolation calculation unit 74 in consideration of reliability. By doing so, the pixel defect correction including the pixel defect of the CCD 42 detected in the initial state before shipment from the factory is performed.

  9, 11, and 14, a control program for causing the CPU of the computer to perform the processes shown in the flowcharts is recorded on a computer-readable recording medium, and the program is stored in the program. The present invention can also be implemented by causing the computer to control the microscope system shown in FIG. 1 by reading it from a recording medium into a computer and causing it to be executed by a CPU.

  Examples of the recording medium from which the recorded control program can be read by a computer include, for example, a storage device such as a ROM or a hard disk device provided as an internal or external accessory device in the computer, a flexible disk, and an MO (magneto-optical disk) A portable recording medium that can be read by a medium driving device provided in a computer, such as a CD-ROM or a DVD-ROM, can be used.

  Such a recording medium may be a storage device provided in a computer functioning as a program server connected to the computer via a network line. In this case, the transmission signal obtained by modulating the carrier wave with the data signal representing the control program is transmitted from the program server to the computer through the network line as the transmission medium, and the computer demodulates the received transmission signal. By reproducing the control program, the control program can be executed by the CPU in the computer.

  In addition, the present invention is not limited to the above-described embodiments, and various improvements and changes can be made without departing from the scope of the present invention.

It is a figure which shows the structure of the microscope system which implements this invention. It is a figure which shows the structure of the electronic camera in FIG. It is a figure which shows the 1st example of a detailed structure of a defect correction circuit. It is a figure which shows the structure of the data stored in a defect memory. It is a figure which shows the rank of a defect shape. It is a figure explaining the interpolation method of a defective pixel. It is a figure which shows the example of the interpolation pattern of a defective pixel. It is a figure which shows the 2nd example of a detailed structure of a defect correction circuit. It is a flowchart which shows the content of the process of a pixel defect detection and correction | amendment performed by CPU. It is a figure which shows the 3rd example of a detailed structure of a defect correction circuit. It is a flowchart which shows the processing content of the interpolation method determination process in case an adjacent defect may arise. It is a figure explaining the interpolation method of the defective pixel in case the adjacent defect has arisen. It is a figure explaining the interpolation method of the defective pixel in a color pixel. It is a flowchart which shows the processing content of the interpolation method determination process in the case of making a color pixel into object. It is a figure explaining the interpolation method of the defective pixel which is a color pixel in case the adjacent defect has arisen. It is a figure which shows the 4th example of a detailed structure of a defect correction circuit. It is a top view which shows an example of the element structure of a CCD image pick-up element. It is sectional drawing of CCD. It is a figure which shows the change of the potential of the depth direction of a photodiode. It is a figure which shows the structural example of a substrate bias generation circuit. It is a figure explaining the example of the image interpolation process which correct | amends a pixel defect. It is a figure which shows the example of the dependence of white spot noise.

Explanation of symbols

S, S ', S "Observation optical path 1 Microscope body 3 Sample 5 Trinocular tube unit 6 Eyepiece unit 6a Eyepiece 12 Epi-illumination optical system 21 Epi-illumination light source 26 Sample stage 27 Objective lens 28 Revolver 30 Cube unit 31 Beam splitter 32 Dichroic mirror 36 Electronic camera 37 Microscope drive circuit 41 Microscope control unit 42 Solid-state imaging device (CCD)
43 CDS circuit 44 Amplifier 45 A / D converter 46 Image memory 47 Temperature sensor 48 CCDSUB voltage switching circuit 49 SUB pulse superposition circuit 51 Image signal processing circuit 52 Light-blocking leak detection circuit 53 Timing generator (TG)
54 Signal Generator (SG)
55 Memory controller 56 DRAM
57 Compression / Expansion Circuit 58 Recording Medium 59 Liquid Crystal Display (LCD)
60 CPU
61 Operation Unit 62 Defect Detection Circuit 63 Defect Memory 64 Defect Correction Circuit 71 RAM
72, 77 Defect position detection unit 73, 76 Interpolation method calculation unit 74 Interpolation calculation unit 75 Defect shape calculation unit 78 Priority selection unit 80 Electronic computer 81 External storage device 100 Imaging lens unit 100a Imaging lens 200 Substrate bias generation circuit 201 Photo Diode 202 Vertical transfer path 203 Horizontal transfer path 204 Signal detector 205 Amplifier 206 Internal VSUB generation circuit 300 n-type semiconductor substrate 301 First area 302 Second area 303 Photoelectric conversion area (charge storage section)
304 Interposition channel 305 Transfer electrode 306 Insulating layer 307 Channel stop region 308 Transfer gate (TG) region 309 Metal layer 311 Substrate bias voltage VSUB

Claims (12)

Of the pixels constituting the image picked up by the image sensor, the pixel defect among the pixels adjacent to the pixel in which the pixel defect has occurred is based on the shape of adjacent pixels having pixel defects caused by the blooming phenomenon of the image sensor. A correction method determining means for determining a correction method for the pixel defect using a non-occurring pixel ;
In accordance with the correction method according to the determination, a correction unit that corrects a pixel defect of the image sensor on an image captured by the image sensor;
An image correction apparatus comprising: A storage unit that stores defect information that is information on pixel defects of the image sensor;
The correction method determining means obtains the position of the pixel having a pixel defect from the defect information.
The image correction apparatus according to claim 1.   The image correction apparatus according to claim 1, further comprising a defect detection unit that detects a pixel defect of the image sensor and generates the defect information.   The image correction apparatus according to claim 1, wherein the determination by the correction method determination means is a selection of a pixel that obtains a luminance value as a basis for calculating a luminance interpolation value of a pixel having a pixel defect. Defect information generating means for generating defect information, which is information relating to pixel defects of the image sensor, based on at least one of an operating temperature and an exposure time of the image sensor when an image is captured by the image sensor. And
The correction method determining means obtains the position of the pixel having a pixel defect from the defect information.
The image correction apparatus according to claim 1. When the position of the pixel having the pixel defect acquired from the defect information is overlapped in the separate pixel, the correction method determination unit eliminates the overlap and corrects the correction based on the shape. the image correction apparatus according to claim 2 or 5, characterized in that determining how. The image pickup device picks up a color image using a Bayer array,
The correction determination unit is configured based on a shape formed by adjacently forming a plurality of pixels having a pixel defect among components constituting an image captured by the image sensor and having the same color component. Determine how to correct pixel defects,
The image correction apparatus according to claim 1. Determined by the correction method determination unit, according to claim 7, characterized in that the selection of the pixels of the same color component to obtain the luminance value of the basis for calculating the luminance interpolation value of pixels having pixel defects Image correction device. It further comprises defect detection means for detecting a pixel defect of the image sensor from a dark signal of the image sensor and generating defect information which is information relating to the pixel defect,
The correction method determining means obtains the position of the pixel having a pixel defect from the defect information.
The image correction apparatus according to claim 1. The dark signal is a signal acquired for performing a subtraction process from a light signal of the image sensor in noise reduction for reducing noise included in an image captured by the image sensor. the image correction apparatus according to claim 9. Of the pixels constituting the image picked up by the image sensor, the pixel defect among the pixels adjacent to the pixel in which the pixel defect has occurred is based on the shape of adjacent pixels having pixel defects caused by the blooming phenomenon of the image sensor. Determining a correction method for the pixel defect using non-occurring pixels ;
In accordance with the correction method according to the determination, the pixel defect of the image sensor is corrected for the image captured by the image sensor.
An image correction method characterized by the above. Of the pixels constituting the image picked up by the image sensor, the pixel defect among the pixels adjacent to the pixel in which the pixel defect has occurred is based on the shape of adjacent pixels having pixel defects caused by the blooming phenomenon of the image sensor. A process for determining a correction method for the pixel defect using a non-occurring pixel ;
In accordance with the correction method according to the determination, a process of correcting pixel defects of the image sensor on an image captured by the image sensor;
A program that causes a computer to perform

Images