JP2010161663A - Fixed pattern noise elimination unit, imaging unit, and electronic endoscope system - Google Patents

Abstract

Fixed pattern noise is removed with high accuracy.
A timing controller causes an image sensor to generate a plurality of image signals with an exposure time of 100 ns to 500 ns. The A / D converter 25 converts the image signal into image data. The image processing unit 30 includes an averaging circuit 31, a determination circuit 32, and an FPN correction circuit 33. The averaging circuit 31 averages image signals having the same exposure time for each pixel. The determination circuit 32 determines whether the difference between the maximum value and the minimum value of the averaged pseudo black pixel data averaged is less than a threshold value. When the difference is less than the threshold value, the average pseudo black pixel data of the minimum data level is stored in the DRAM 27 as noise data. The FPN correction circuit 33 removes FPN from the imaging pixel data using the noise data.
[Selection] Figure 2

Description

  The present invention relates to a fixed pattern noise removing unit that effectively removes fixed pattern noise that differs from pixel to pixel with a simple configuration, such as a CMOS image sensor, an imaging unit, and an electronic that creates an image from which fixed pattern noise has been removed. The present invention relates to an endoscope system.

  A CMOS image sensor that can reduce power consumption and manufacturing cost is known. Since the CMOS image sensor is provided with an amplifier for each pixel, individual fixed pattern noise is generated for each pixel.

  In addition, in order to remove fixed pattern noise, the exposure time is set short and the electronic shutter functions so that fixed pattern noise is generated by imaging under the same conditions as the light-shielded state, and is fixed from the image data at the time of imaging. It has been proposed to remove pattern noise components (see Patent Document 1).

  However, when the exposure time cannot be set sufficiently short, or when a large amount of light is incident, it is not possible to generate highly accurate fixed pattern noise. Therefore, under such circumstances, it has been difficult to remove fixed pattern noise with high accuracy.

JP 10-313428 A

  Accordingly, an object of the present invention is to provide a fixed pattern noise removing unit that effectively removes fixed pattern noise with a simple configuration.

  The fixed pattern noise removal unit of the present invention includes a receiving unit that receives a pixel signal generated by a plurality of pixels arranged in an image sensor according to the amount of received light, and an exposure time of pixels in the image sensor as a first exposure time or By switching to the second to mth (m is an integer of 3 or more) exposure times that are shorter than the first exposure time and different from each other, the imaging pixel signal or the second pixel signal corresponding to the amount of received light in the first exposure time An image sensor control unit that generates a pseudo black pixel signal that is a pixel signal corresponding to the amount of received light in the m-th exposure time, and second to second in accordance with the amount of received light in the second to m-th exposure time in the same pixel. A determination unit configured to determine whether the second pseudo black pixel signal can be regarded as fixed pattern noise in the corresponding pixel based on the mth pseudo black pixel signal; The noise signal based on the considered has been the second pseudo black pixel signal is characterized in that it comprises a subtraction unit for subtracting from the image pixel signal.

  Further, the image sensor control unit causes the image sensor to perform imaging for the second to m-th exposure times a plurality of times to generate a plurality of 2 to m-th pseudo black pixel signals for each pixel, It is preferable that the noise signal based on the second pseudo black pixel signal regarded as the fixed pattern noise by the determination unit among the plurality of second pseudo black pixel signals is subtracted from the imaging pixel signal.

  In addition, the subtracting unit is a plurality of second pseudo black pixel signals that are a plurality of second pseudo black pixel signals generated by imaging at a plurality of times of the second exposure time and are regarded as fixed pattern noise by the determination unit. Preferably, the signal is averaged for each pixel, and the averaged second pseudo black pixel signal is subtracted from the imaged pixel signal for each pixel as a noise signal.

  The subtracting unit averages the second pseudo black pixel signal regarded as the fixed pattern noise by the discrimination unit for each row of pixels, and the averaged second pseudo black pixel signal for each row is a noise signal. Is preferably subtracted from the imaging pixel signal in the corresponding row.

  In addition, the determination unit regards the second pseudo black pixel signal as fixed pattern noise when the difference between the maximum value and the minimum value of the signal levels of the second to mth pseudo black pixel signals is less than the threshold value. Is preferred.

  In addition, the subtracting unit is the difference between the maximum value and the minimum value of the signal levels of the second to mth pseudo black pixel signals among the second pseudo black pixel signals regarded as fixed pattern noise in each row or each column. It is preferable to average a predetermined number of second pseudo black pixel signals in ascending order.

  In addition, the image sensor control unit switches the pixel exposure time to the second to m-th exposure times when the image pickup apparatus provided with the fixed pattern noise removal unit is started, and generates the second to m-th pseudo black pixel signals. It is preferable.

  The image sensor control unit switches the pixel exposure time to the second to m-th exposure time while continuously generating the image pickup pixel signal by switching the pixel exposure time to the first exposure time. It is preferable to generate the second to mth pseudo black pixel signals.

  Further, an input unit for inputting a command for generating the second to m-th pseudo black pixel signals is provided, and the imaging element control is performed when a command for generating the second to m-th pseudo black pixel signals is input to the input unit. The unit preferably switches the pixel exposure time to the second to m-th exposure times to generate the second to m-th pseudo black pixel signals.

  In addition, it is preferable that the second to m-th exposure times are set to be less than a time in which the integrated exposure amount to the pixel can be regarded as an integrated exposure amount corresponding to black. Alternatively, the second exposure time is preferably the shortest exposure time that can be set in the image sensor.

  An image pickup unit according to the present invention includes an image pickup device having a plurality of pixels that generate a pixel signal corresponding to the amount of received light, and a first exposure time or a first exposure time shorter than the first exposure time. By switching to the exposure time of 2 to mth (m is an integer of 3 or more), the imaging pixel signal which is a pixel signal corresponding to the amount of received light in the first exposure time or the amount of received light in the 2nd to mth exposure time. Based on the image sensor control unit for generating a pseudo black pixel signal that is a corresponding pixel signal, and the second to m-th pseudo black pixel signals corresponding to the received light amount in the second to m-th exposure time in the same pixel A determination unit that determines whether or not the second pseudo black pixel signal can be regarded as fixed pattern noise in the corresponding pixel, and a second pseudo black pixel signal that is regarded as fixed pattern noise by the determination unit. It is characterized in that it comprises a subtraction unit for subtracting from the imaging pixel signal Ku noise signal.

  The electronic endoscope system according to the present invention includes an imaging device having a plurality of pixels that generate a pixel signal corresponding to the amount of received light, and the exposure time of the pixels in the imaging device is shorter than the first exposure time or the first exposure time. By switching to different second to m-th exposure times (m is an integer of 3 or more), an imaging pixel signal that is a pixel signal corresponding to the amount of received light in the first exposure time or light reception in the second to m-th exposure time Based on the second to m-th pseudo black pixel signals corresponding to the received light amounts in the second to m-th exposure times in the same pixel as the image sensor control unit that generates a pseudo-black pixel signal that is a pixel signal corresponding to the amount. A determination unit that determines whether the second pseudo black pixel signal can be regarded as fixed pattern noise in the corresponding pixel, and a second pseudo black pixel signal that is regarded as fixed pattern noise by the determination unit. It is characterized by having an imaging unit and a subtraction unit subtracting the Ku noise signals from the image pixel signal.

  According to the present invention, it is possible to remove fixed pattern noise mixed in an image signal of an image sensor with high accuracy with a simple configuration.

1 is a block diagram schematically showing an internal configuration of an electronic endoscope system having a fixed pattern noise removal unit to which a first embodiment of the present invention is applied. FIG. It is a block diagram which shows roughly the internal structure of the image process part of 1st Embodiment. 1 is a block diagram schematically showing an internal configuration of a DRAM of a first embodiment. FIG. It is a flowchart for demonstrating the process of the removal of the FPN component in 1st Embodiment. It is a flowchart for demonstrating the subroutine of noise data generation in 1st Embodiment. It is a block diagram which shows roughly the internal structure of the image process part of 2nd Embodiment. It is a block diagram which shows roughly the internal structure of DRAM of 2nd Embodiment. It is a flowchart for demonstrating the process of the removal of FPN component in 2nd Embodiment. It is a flowchart for demonstrating the subroutine of noise data generation in 2nd Embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram schematically showing an internal configuration of an electronic endoscope system having a fixed pattern noise removing unit to which the first embodiment of the present invention is applied.

  The electronic endoscope system 10 includes an electronic endoscope 20, an endoscope processor 40, and a monitor 11. The endoscope processor 40 is connected to the electronic endoscope 20 and the monitor 11.

  The endoscope processor 40 is provided with a light source unit (not shown). Illumination light emitted from the light source unit is transmitted to the distal end of the insertion tube 22 by a light guide (not shown) extending from the connector 21 of the electronic endoscope 20 to the distal end of the insertion tube 22.

  Illumination light transmitted by the light guide is irradiated near the tip of the insertion tube 22. The subject irradiated with the illumination light is imaged by the electronic endoscope 20. An image signal generated by imaging of the electronic endoscope 20 is sent to the endoscope processor 40.

  The endoscope processor 40 includes an image processing circuit 41, and predetermined signal processing is performed on an image signal obtained from the electronic endoscope 20 in the image processing circuit 41. The image signal subjected to the predetermined signal processing is transmitted to the monitor 11, and an image corresponding to the transmitted image signal is displayed on the monitor 11.

  The operation of the light source unit and image processing circuit 41 is controlled by a system controller 42 provided in the endoscope processor 40. The system controller 42 is also connected to the electronic endoscope 20 and controls the operation of each part of the electronic endoscope 20.

  Note that the endoscope processor 40 is provided with an input unit 43. When a command from the user is input, an order signal corresponding to the input command is transmitted from the input unit 43 to the system controller 42. The system controller 42 controls the operation of each part according to the received order signal.

  The electronic endoscope 20 includes an objective lens 23, an image sensor 24, an A / D converter 25 (receiver), an image processor 30, a D / A converter 26, a DRAM 27, a timing controller 28 (image sensor controller), and the like. Provided.

  As described above, the optical image of the subject irradiated with the illumination light reaches the light receiving surface of the image sensor 24 through the objective lens 23. Based on the driving of the timing controller 28, the image sensor 24 generates an image signal corresponding to the optical image that has reached the light receiving surface.

  The image sensor 24 is a CMOS image sensor, and a plurality of pixels (not shown) are arranged in a matrix on the light receiving surface. In each pixel, a pixel signal corresponding to the amount of received light is generated.

  The pixel signal generated in each pixel has a fixed pattern noise (FPN) component and a received light amount component. The FPN component is a noise component unique to each pixel, and has a constant magnitude regardless of the exposure time. The received light amount component changes according to the integrated exposure amount to the pixel, that is, the product of the amount of incident light and the exposure time.

  The output timing of the pixel signal generated by each pixel is controlled by the timing controller 24, and the pixel signal is sequentially output from the image sensor 24. An image signal of one frame is constituted by pixel signals generated by all the pixels arranged on the light receiving surface.

  As will be described later, the timing controller 28 can also control the exposure time of the pixels. Further, the timing controller 28 also controls the operation timing in the A / D converter 25 and the image processing unit 30.

  The generated image signal is transmitted from the image sensor 24 to the A / D converter 25. A / D conversion is performed by the A / D converter 25, and the analog image signal is converted into digital image data. The image data is constituted by pixel data obtained by A / D converting the pixel signal.

  The image data is transmitted to the image processing unit 30. In the image processing unit 30, predetermined image processing including FPN removal is performed on the image data. Note that the image processing unit 30 is connected to the DRAM 27. The DRAM 27 stores pixel data for FPN removal, as will be described later.

  Image data that has undergone predetermined image processing is transmitted to the D / A converter 26. D / A conversion is performed by the D / A converter 26, and the digital image data is converted into an analog image signal. The converted image signal is transmitted to the image processing circuit 41 of the endoscope processor 40. As described above, the image processing circuit 41 performs predetermined image processing on the image signal. After the image processing, the image signal is transmitted to the monitor 11 and an image corresponding to the image signal is displayed on the monitor 11.

  Next, the configuration of the image processing unit 30 and the FPN removal executed in the image processing unit 30 will be described. As shown in FIG. 2, the image processing unit 30 includes an averaging circuit 31 (subtraction unit), a determination circuit 32, an FPN correction circuit 33 (subtraction unit), a general image processing circuit 34, and the like.

  In order to remove the FPN component from the pixel signal, a noise signal that is an FPN component unique to each pixel is required. Prior to the FPN removal, noise data is generated by the image processing unit 30.

  In order to generate noise data, when the electronic endoscope 20 is activated, the timing controller 28 causes the imaging device 24 to perform 10 imaging operations with the shortest exposure time that can be set, for example, an exposure time of 100 ns, and the image sensor 24 has 10 frames of images. Generate a signal. When the exposure time is 100 ns, the generated pixel signal is transmitted to the averaging circuit 31 as pseudo black pixel data.

  The averaging circuit 31 is connected to the DRAM 27. As shown in FIG. 3, in the DRAM 27, an averaging calculation storage area 27a, an averaged image storage area 27b, and an FPN storage area 27c are set.

  The pseudo black pixel data received by the averaging circuit 31 is stored in the averaging calculation storage area 27a. The averaging calculation storage area 27a can store image data of a plurality of frames, and the pseudo black pixel data is an area determined in the averaging calculation storage area 27a according to the order of the frames and the position of the corresponding pixel. Stored in

  When the pseudo black pixel data of all 10 frames with an exposure time of 100 ns are stored in the averaging calculation storage area 27a, 10 pseudo black pixel data having different frames are read out to the averaging circuit 31 for each pixel. In the averaging circuit 31, the average pseudo black pixel data for each pixel is generated by averaging the data components.

  100ns averaged image data is generated by generating corresponding average pseudo black pixel data for all pixels. The 100 ns averaged image data is transmitted to and stored in the averaged image storage area 27b. The averaged image storage area 27b includes areas for separately storing 100ns averaged image data, 200ns averaged image data, 300ns averaged image data, 400ns averaged image data, and 500ns averaged image data. Yes.

  When the 100 ns averaged image data is generated, the timing controller 28 causes the imaging device 24 to generate an image signal of 10 frames by causing the imaging device 24 to perform imaging 10 times with an exposure time of 200 ns. Thereafter, similarly to the case of 100 ns, 200 ns averaged image data is generated and stored in the averaged image storage area 27b.

  Similarly, 300 ns averaged image data, 400 ns averaged image data, and 500 ns averaged image data corresponding to exposure times of 300 ns, 400 ns, and 500 ns are generated and stored in the averaged image storage area 27b.

  When all the 100 ns to 500 ns averaged image data is stored in the averaged image storage area 27b, the average pseudo black pixel data corresponding to the same pixel among the average pseudo black pixel data constituting the 100 ns to 500 ns averaged image data. Is read by the determination circuit 32.

  The discriminating circuit 32 discriminates the maximum value and the minimum value of the data level among the read five average pseudo black pixel data. Next, it is determined whether or not the difference between the maximum value and the minimum value is less than a threshold value. When the difference is less than the threshold, the average pseudo black pixel data having the minimum data level is selected as the noise data of the corresponding pixel. The selected noise data is transmitted from the discrimination circuit 32 to the FPN storage area 27c and stored.

  If the difference between the maximum value and the minimum value is not less than the threshold value, no average pseudo black pixel data is selected. When there is a pixel for which noise data is not selected, imaging with an exposure time of 100 ns to 500 ns, generation of average image data of 100 ns to 500 ns, and determination of the difference between the maximum value and the minimum value are performed again for all pixels. Repeat until noise data is selected. When noise data corresponding to all pixels is selected and stored in the FPN storage area 27c, the generation of noise data is terminated.

  The pseudo black pixel data, that is, the pixel data includes the FPN component and the received light amount component as described above. As described above, the received light amount component has a magnitude corresponding to the product of the amount of incident light and the exposure time. Therefore, if the incident light amount and the exposure time are sufficiently short, a pseudo black pixel signal is generated in which the signal level of the received light amount component can be regarded as zero. However, even if the exposure time is sufficiently shortened, if the amount of incident light is large, a pseudo black pixel signal including a received light amount component having a signal level greater than zero is generated.

  When the pseudo black pixel signal includes a received light amount component having a signal level greater than zero, the received light amount component changes according to the exposure time. Conversely, if the magnitude of the signal level of the pseudo black pixel signal does not change substantially with respect to the change in the exposure time, the signal level of the received light amount component can be regarded as substantially zero.

  Therefore, in the discrimination circuit 32, if the maximum value and the minimum value of the average pseudo black pixel data with sufficiently short and different exposure times are substantially the same, that is, the difference between the two is less than the threshold value, the image sensor It can be determined that the amount of light incident on 24 is sufficiently small and the pseudo black pixel signal substantially contains only the FPN component.

  When the generation of the noise data is finished, the timing controller 28 causes the image sensor 24 to generate an image signal with an exposure time for capturing a normal image, for example, an exposure time of 1/60 s. Each pixel signal constituting one frame of image signal is sequentially transmitted to the image processing unit 30 via the A / D converter 25 as an imaging pixel signal.

  When the exposure time is 1/60 s, the generated pixel data that has been A / D converted is transmitted to the FPN correction circuit 33. The FPN correction circuit 33 reads out noise data of the pixel corresponding to the received imaging pixel data from the DRAM 27. The FPN component is removed by subtracting noise data from the imaged pixel data by the FPN correction circuit 33.

  The corrected imaging pixel data from which the FPN component has been removed is transmitted to the general image processing circuit 34 and subjected to predetermined image processing such as color interpolation processing and gamma correction processing. The corrected imaging pixel data subjected to the predetermined image processing is transmitted to the D / A converter 26 as described above.

  Next, noise data generation and FPN component removal processing from the imaged pixel data performed by the timing controller 28 and the image processing unit 30 will be described with reference to the flowcharts of FIGS. 4 and 5.

  FIG. 4 is a flowchart for explaining processing for removing the FPN component. FIG. 5 is a flowchart for explaining a subroutine for generating noise data. The process of removing the FPN component is started when the electronic endoscope system 10 is turned on.

  When the electronic endoscope 20 is activated in step S100, the process proceeds to step S200, and noise data is generated for each pixel. When the generation of noise data ends, the process proceeds to step S101.

  In step S101, the image pickup device 24 picks up an image of a subject and generates a one-frame image signal. As will be described later, the exposure time of the image sensor 24 is set to 1/60 s. When one frame image signal is generated, the process proceeds to step S102.

  In step S102, the FPN component is removed from the image signal generated in step S101. That is, the noise data stored in the DRAM 27 in step S200 is subtracted from each imaging pixel data constituting the image data obtained by A / D converting the image signal, thereby removing the FPN component unique to each imaging pixel data. When the FPN component of all the imaging pixel data is removed, the process proceeds to step S103.

  In step S103, predetermined image processing such as color interpolation processing and gamma processing is performed on the image data from which the FPN component has been removed. When predetermined image processing is performed, the process proceeds to step S104, and it is determined whether or not a command for ending the moving image display is input.

  When the command for ending the moving image display has not been input, the process returns to step S101, and steps S101 to S104 are repeated until the command for ending the moving image display is input. When the command for ending the moving image display is input, the FPN component removal processing is ended.

  Next, a noise data generation subroutine will be described. As described above, after activation of the electronic endoscope 20 (step S100), a noise data generation subroutine is started. In step S201, the exposure time of the image sensor 24 is set to 100 ns. After setting the exposure time of the image sensor 24, the process proceeds to step S202.

  In step S202, the image sensor 24 is caused to image the subject ten times to generate a 10-frame image signal. The generated image signal is A / D converted and stored in the DRAM 27. After storing in the DRAM 27, the process proceeds to step S203.

  In step S203, the averaged image data is generated by averaging the pseudo black pixel data constituting the image data of 10 frames for each pixel. Further, the generated averaged image data is stored in the DRAM 27, and the process proceeds to step S204.

  In step S204, it is determined whether or not the exposure time at the time of imaging in the latest step S202 is 500 ns. If not 500 ns, the process proceeds to step S205. In step S205, the exposure time is reset by adding +100 ns to the set exposure time. After the resetting, the process returns to step S202, and the processes in steps S202 to 205 are repeated until the exposure time is determined to be 500 ns in step S204. If the exposure time is 500 ns in step S204, the process proceeds to step S206.

  In step S205, the average pseudo black pixel data corresponding to the specific pixel is read out from the average pseudo black pixel data constituting the 100 ns to 500 ns averaged image data stored in the DRAM 27. After reading, the process proceeds to step S207.

  In step S207, average pseudo black pixel data having a maximum and minimum data level is selected from the five average pseudo black pixel data read out. Next, it is determined whether or not the difference in data level of the selected data is less than a threshold value. If the data level difference is less than the threshold, the process proceeds to step S208. If the difference in data level is not less than the threshold value, step S208 is skipped and the process proceeds to step S209.

  In step S208, the average pseudo black pixel data having the minimum data level is stored in the DRAM 27 as noise data. After storing in the DRAM 27, the process proceeds to step S209.

  In step S209, it is determined whether or not the determination in step S207 has been performed for all pixels. If not, the process proceeds to step S210, and average pseudo black pixel data of a pixel different from the pixel corresponding to the read average pseudo black pixel data is read from the DRAM 27. After reading, the process returns to step S207. Thereafter, the processing in steps S207 to S210 is repeated until all the pixels are discriminated. If discrimination is made for all pixels, the process proceeds to step S211.

  In step S211, it is determined whether noise data for all pixels is stored in the DRAM 27 or not. If noise data corresponding to all pixels is not stored, the process returns to step S201. Thereafter, the processing from step S201 to step S211 is repeated until noise data for all the pixels is stored in the DRAM 27 in step S211. If noise data for all pixels is stored, the process proceeds to step S212.

  In step S212, the exposure time of the image sensor 24 is set to 1/60 s. After setting the exposure time of the image sensor 24, the process proceeds to step S101 as described above.

  According to the fixed pattern noise elimination unit to which the first embodiment as described above is applied, it is possible to remove the FPN with high accuracy with a simple configuration.

  Next, a fixed pattern noise removing unit to which the second embodiment of the present invention is applied will be described. The second embodiment differs from the first embodiment in generated noise data. Hereinafter, the second embodiment will be described with a focus on portions different from the first embodiment. In addition, the same code | symbol is attached | subjected to the site | part which has the same function.

  In general, the signal level of the FPN component unique to each pixel tends to change depending on the column in which the pixels are arranged, and the FPN components of the pixels arranged in the same column are substantially equal. Therefore, unlike the first embodiment, in this embodiment, column direction noise data is generated for each column and used to remove the FPN component.

  Configurations and functions of parts other than the image processing unit and DRAM of the electronic endoscope of the second embodiment are the same as those of the electronic endoscope of the first embodiment. In the second embodiment, the endoscope processor and the monitor are the same as the endoscope processor and the monitor in the first embodiment.

  A description will be given of FPN removal executed in the image processing unit, along with the configuration and functions of the image processing unit of the second embodiment. As shown in FIG. 6, the image processing unit 300 includes an averaging circuit 31, a line noise creation circuit 35 (subtraction unit), an FPN correction circuit 33, a general image processing circuit 34, and the like.

  Similar to the first embodiment, column direction noise data for each column is generated by the image processing unit 300 before the FPN removal.

  Similar to the first embodiment, when the electronic endoscope 20 is started, imaging is performed while changing the exposure time to 100 ns, 200 ns, 300 ns, 400 ns, and 500 ns, thereby obtaining 100 ns averaged image data and 200 ns averaged image. Data, 300 ns averaged image data, 400 ns averaged image data, and 500 ns averaged image data are generated and stored in the averaged image storage area 27b (see FIG. 7) of the DRAM 270.

  The average pseudo black pixel data constituting the stored averaged image data is read out to the line noise generating circuit 35 for each pixel. As with the determination circuit 32 in the first embodiment, the line noise generation circuit 35 determines whether the difference between the maximum value and the minimum value of the data level among the five average pseudo black pixel data read is less than the threshold value. Determine whether or not.

  Unlike the discrimination circuit 32 of the first embodiment, the line noise creation circuit 35 erases the averaged pseudo black pixel data of pixels whose difference between the maximum value and the minimum value is not less than the threshold from the averaged image storage area 27b. .

  When the comparison of the threshold value of the difference between the maximum value and the minimum value of the data level is completed for all the pixels, the line noise generation circuit 35 does not delete the number of pixels stored in the averaged image storage area 27b. Measure for each column. Until the number of pixels reaches 100 pixels, generation of 100 ns to 500 ns averaged image data and determination of whether or not to erase the average pseudo black pixel data are executed.

  The line noise generation circuit 35 generates column direction noise data for a column having 100 or more pixels corresponding to the average pseudo black pixel data stored without being erased. 100 pixels are selected as pixels for generating noise data in order of increasing difference in data level between the maximum value and the minimum value. The average pseudo black pixel data having the minimum data level corresponding to the selected pixel is read from the averaged image storage area 27b and averaged. Column direction noise data is generated by the averaging and stored in the line noise storage area 27d.

  When the column direction noise data corresponding to all the columns is stored, the timing controller 28 causes the image sensor 24 to generate an image signal with an exposure time of 1/60 s, as in the first embodiment. In addition, each imaging pixel signal constituting one frame of image signal is sequentially transmitted to the image processing unit 300 via the A / D converter 25.

  As in the first embodiment, when the exposure time is 1/60 s, the generated and A / D converted image pixel data is transmitted to the FPN correction circuit 33. Unlike the first embodiment, the FPN correction circuit 33 reads out the column direction noise data of the column of pixels corresponding to the received imaging pixel data from the DRAM 270, and removes the FPN component by subtracting the column direction noise data from the imaging pixel data. Is done.

  Next, noise data creation processing and FPN component removal processing from the imaging pixel data performed by the image processing unit 300 will be described with reference to the flowcharts of FIGS.

  FIG. 8 is a flowchart for explaining the process of removing the FPN component. FIG. 9 is a first flowchart for explaining a subroutine of noise data generation. The process of removing the FPN component is started when the electronic endoscope system 10 is turned on.

  When the electronic endoscope 20 is activated in step S300, the process advances to step S400 to generate column direction noise data for each column in which pixels are arranged. When the generation of the column direction noise data ends, the process proceeds to step S301. Thereafter, the same processing as Steps S101 to S104 in the first embodiment is executed in Steps S301 to S304.

  Next, a noise data generation subroutine will be described. As in the first embodiment, after the electronic endoscope 20 is activated (step S300), a noise data generation subroutine is started. In steps S401 to S406, the same processing as in steps S201 to S206 in the first embodiment is executed.

  In step S407, as in step S207 in the first embodiment, average pseudo black pixel data whose data level is the maximum value and the minimum value is selected from the five average pseudo black pixel data read out. Next, it is determined whether or not the difference between the selected data levels is less than a threshold value. If the difference in data level is not less than the threshold value, the process proceeds to step S408. If the difference in data level is less than the threshold value, step S408 is skipped and the process proceeds to step S409.

  In step S408, the average pseudo black pixel data whose data level difference exceeds the threshold value is erased from the averaged image storage area 27b. After erasing, the process proceeds to step S409.

  In step S409, as in step S209 in the first embodiment, it is determined whether or not the determination in step S407 has been performed for all pixels. If not, the process proceeds to step S410, and average pseudo black pixel data of a pixel different from the pixel corresponding to the read average pseudo black pixel data is read from the DRAM 270. After reading, the process returns to step S407. Thereafter, the processes in steps S407 to S410 are repeated until all the pixels are discriminated. If discrimination has been performed for all pixels, the process proceeds to step S411.

  In step S411, it is determined whether or not there are 100 or more pixels corresponding to the average pseudo black pixel data that has not been erased for each column. If there are not 100 or more pixels in each column, the process returns to step S401, and thereafter, until the average pseudo black pixel data corresponding to the 100 or more pixels is stored without being erased in the column in which there are no 100 or more pixels. The processing from S401 to step S411 is repeated. If there are 100 or more pixels in each column, the process proceeds to step S412.

  In step S412, the average pseudo black pixel data having the minimum data level corresponding to 100 pixels is read out in order of increasing difference between the maximum and minimum data levels, and averaged to generate column noise data. When the column direction noise data of all the columns is generated, the process proceeds to step S413.

  In step S413, similarly to step S212 in the first embodiment, the exposure time tube of the image sensor 24 is set to 1/60 s. After setting the exposure time of the image sensor 24, the process proceeds to step S301 as described above.

  According to the fixed pattern noise elimination unit to which the second embodiment as described above is applied, it is possible to remove the FPN with high accuracy as in the first embodiment.

  Further, according to the present embodiment, noise data is not generated for all pixels, so that it is possible to measure the speed of noise data generation and the reduction in capacity of the DRAM 270.

  Even in the case where a large amount of light is incident on a part of the image sensor 24, column direction noise data can be generated using pseudo black pixel signals in other pixels in the same column. Therefore, even in a situation where noise data cannot be generated in the first embodiment, it is possible to generate noise data necessary for FPN removal of image data.

  In the first and second embodiments, the noise data is generated using the image signal generated with the exposure time of 5 stages of 100 ns to 500 ns, but the exposure time is not limited to 5 stages. Even if noise data is generated using a plurality of image signals generated at least in two or more exposure times, the same effect as in the present embodiment can be obtained.

  However, since the position of the image sensor 24 with respect to the subject is not fixed, the incident optical image may be shifted depending on the time when the image is captured. When the exposure time stage of the image data to be used is small, the difference in the data level of the average pseudo black pixel data may be less than the threshold due to the shift of the optical image, and the accuracy of the noise data may be lowered. On the other hand, as in this embodiment, it is possible to eliminate the influence due to the shift of the optical image by increasing the number of exposure steps.

  In the first and second embodiments, when the difference between the maximum value and the minimum value of the average pseudo black pixel data for the same pixel is less than the threshold value, the average pseudo black pixel data is regarded as noise data. Although the configuration is considered, the average pseudo black pixel data may be regarded as noise data by another method based on a plurality of average pseudo black pixel data having different exposure times for the same pixel.

  In the first and second embodiments, the average pseudo black pixel data having the minimum data level is handled as noise data in the pixel in which the difference between the maximum value and the minimum value of the data level is less than the threshold value. However, if the difference between the maximum value and the minimum value is less than the threshold value, sufficiently accurate noise data can be generated even if the average pseudo black pixel data of other exposure times is used as the noise data.

  Further, in the first and second embodiments, the imaging is repeated 10 times in each of the exposure time of 100 ns to 500 ns. However, the imaging may be performed once in each. However, it is preferable to generate averaged image data by performing imaging a plurality of times as in the present embodiment in order to increase the accuracy of noise data.

  Further, in the second embodiment, the average pseudo black pixel data of 100 pixels is used for generating the column direction noise data in the order from the smallest difference between the maximum value and the minimum value in the same column. The number of pixels to be processed is not limited to 100. In addition, pixels used for generating the column direction noise data may be selected by other methods. For example, the average pseudo black pixel data of 100 pixels may be selected in order from the lowest data level.

  In the first and second embodiments, noise data is generated and stored in the DRAMs 27 and 270 immediately after the electronic endoscope 20 is started, but immediately after the electronic endoscope 20 is started, Noise data generation may be started automatically during observation of the subject. Furthermore, generation of noise data may be started based on a noise data generation command input to the input unit 43.

  In the first and second embodiments, the image sensor 24 is driven so that one exposure time is generated with the shortest exposure time that can be set in the image sensor 24 for noise data generation. As long as the exposure time can be regarded as a value when the integrated exposure amount is black, the exposure time may not be the shortest. Since the integrated exposure amount is also affected by the luminance of the subject itself, it can be used to generate noise data if the image is taken with an exposure time shorter than an exposure time at which normal photographing is possible.

  In the first and second embodiments, the fixed pattern noise removal unit is configured to be applied to an electronic endoscope system, but may be applied to other imaging devices such as a digital still camera and a digital video camera. Good.

  In the first and second embodiments, the image sensor 24 is a CMOS image sensor. However, the present invention is also applicable to other types of image sensors in which an amplifier is provided in each pixel and fixed pattern noise unique to the pixel occurs. Is possible.

DESCRIPTION OF SYMBOLS 10 Electronic endoscope system 20 Electronic endoscope 24 Image pick-up element 25 A / D converter 27,270 DRAM
27a Average calculation storage area 27b Averaged image storage area 27c FPN storage area 27d Line noise storage area 28 Timing controller 30 Image processing unit 31 Averaging circuit 32 Discriminating circuit 33 FPN correction circuit 35 Line noise creating circuit 40 Endoscope processor 42 System controller 43 Input section

Claims (14)

A receiving unit that receives a pixel signal generated by a plurality of pixels arranged in the imaging device according to the amount of received light;
The first exposure time or the second to mth (m is an integer of 3 or more) exposure times that are shorter than the first exposure time or different from the first exposure time are switched to the first exposure time. An imaging element control unit that generates an imaging pixel signal that is the pixel signal corresponding to the amount of received light during the exposure time or a pseudo black pixel signal that is the pixel signal corresponding to the amount of received light during the second to m-th exposure times; ,
The second pseudo black pixel signal is fixed in the corresponding pixel based on the second to m-th pseudo black pixel signals corresponding to the received light amounts in the second to m-th exposure times in the same pixel. A discriminator for discriminating whether or not it can be regarded as pattern noise;
A fixed pattern noise removing unit, comprising: a subtracting unit that subtracts a noise signal based on the second pseudo black pixel signal regarded as the fixed pattern noise by the determining unit from the imaging pixel signal. The imaging element control unit generates a plurality of the second to m-th pseudo black pixel signals for each pixel by causing the imaging element to perform imaging for the second to m-th exposure times a plurality of times. Let
The subtracting unit converts the noise signal based on the second pseudo black pixel signal, which is regarded as the fixed pattern noise by the determination unit among the plurality of second pseudo black pixel signals, to the imaging pixel signal The fixed pattern noise elimination unit according to claim 1, wherein The subtracting unit is the plurality of second pseudo black pixel signals generated by imaging the second exposure time a plurality of times, and the plurality of second pseudo black pixel signals regarded as the fixed pattern noise by the determination unit. 3. The two pseudo black pixel signals are averaged for each pixel, and the averaged second pseudo black pixel signal is subtracted from the imaging pixel signal for each pixel as the noise signal. Fixed pattern noise elimination unit as described.   The subtracting unit averages the second pseudo black pixel signal, which is regarded as the fixed pattern noise by the discrimination unit, for each row in which the pixels are arranged, and the second pseudo black pixel for each averaged row. The fixed pattern noise removal unit according to claim 1, wherein a pixel signal is subtracted from the imaging pixel signal in a corresponding row as the noise signal.   The determination unit determines that the second pseudo black pixel signal is the fixed pattern noise when the difference between the maximum value and the minimum value of the signal levels of the second to m-th pseudo black pixel signals is less than a threshold value. The fixed pattern noise removing unit according to claim 4, wherein the fixed pattern noise removing unit is regarded.   The subtracting unit includes a maximum value of the signal level of the second to m-th pseudo black pixel signals among the second pseudo black pixel signals regarded as the fixed pattern noise in each row or each column. 6. The fixed pattern noise removing unit according to claim 5, wherein a predetermined number of the second pseudo black pixel signals are averaged in order of increasing difference from the minimum value.   The determination unit determines that the second pseudo black pixel signal is the fixed pattern noise when the difference between the maximum value and the minimum value of the signal levels of the second to m-th pseudo black pixel signals is less than a threshold value. The fixed pattern noise removing unit according to any one of claims 1 to 3, wherein the fixed pattern noise removing unit is considered.   The imaging element control unit switches the exposure time of the pixels to the second to mth exposure times and activates the second to mth pseudo black pixels when starting an imaging apparatus provided with the fixed pattern noise removal unit. 8. The fixed pattern noise removing unit according to claim 1, wherein a signal is generated.   The image sensor control unit switches the exposure time of the pixels to the first exposure time and continuously generates the image pickup pixel signal while changing the exposure time of the pixels to the second to mth. The fixed pattern noise removing unit according to claim 1, wherein the second to m-th pseudo black pixel signals are generated by switching to the exposure time. An input unit for inputting a command for generating the second to mth pseudo black pixel signals;
When a command for generating the second to mth pseudo black pixel signals is input to the input unit, the image sensor control unit switches the exposure time of the pixels to the second to mth exposure times. The fixed pattern noise removing unit according to any one of claims 1 to 7, wherein the second to mth pseudo black pixel signals are generated.   11. The second to m-th exposure times are set to be less than a time period in which an integrated exposure amount to the pixel can be regarded as an integrated exposure amount corresponding to black. The fixed pattern noise removing unit according to item 1.   11. The fixed pattern noise elimination unit according to claim 1, wherein the second exposure time is a shortest exposure time that can be set in the imaging element. An imaging device having a plurality of pixels that generate pixel signals according to the amount of received light;
The first exposure time or the second to mth (m is an integer of 3 or more) exposure times that are shorter than the first exposure time or different from the first exposure time are switched to the first exposure time. An imaging element control unit that generates an imaging pixel signal that is the pixel signal corresponding to the amount of received light during the exposure time or a pseudo black pixel signal that is the pixel signal corresponding to the amount of received light during the second to m-th exposure times; ,
The second pseudo black pixel signal is fixed in the corresponding pixel based on the second to m-th pseudo black pixel signals corresponding to the received light amounts in the second to m-th exposure times in the same pixel. A discriminator for discriminating whether or not it can be regarded as pattern noise;
An imaging unit comprising: a subtracting unit that subtracts, from the imaging pixel signal, a noise signal based on the second pseudo black pixel signal regarded as the fixed pattern noise by the discrimination unit.   An electronic endoscope system comprising the imaging unit according to claim 13.

Images