JP2008311850A - Image processing apparatus and endoscopic instrument equipped with the same - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus which performs effective noise reduction processing, considering the results of motion detection in a field-sequential imaging method, and also to provide an endoscopic instrument equipped with the same. <P>SOLUTION: The image processing apparatus comprises: a motion detecting means 43 for detecting the motion of an object using a first image field signal of each color inputted sequentially by a field-sequential method and a second image field signal of each color which is the first image field signal reduced in noise and stored (42); and a predicted image creating means 46 for creating a predicted image field signal of each color using the detected motion of the object and the second image field signal of the same color as the first image field signal. A noise reducing means used for reducing the noise signal combines the first image field signal inputted at the point of time when it became a target for processing, and the predicted image field signal to perform noise reduction processing 47 on the first image field signal inputted at the point of time when it became a target for processing. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that effectively reduces noise components in a frame sequential imaging method, and an endoscope apparatus including the image processing apparatus.

  There are a frame sequential method and a simultaneous method as an imaging method of an imaging device that displays a subject in color, particularly an electronic endoscope device. When using a frame sequential imaging method, even when the number of pixels is small, the resolution is relatively high and the color reproducibility is excellent, so in the medical field where it is necessary to observe subtle color changes, A type of electronic endoscope apparatus is widely used. In addition, recently, a field sequential type electronic endoscope apparatus capable of obtaining a narrow band light observation image using narrow band illumination light is also known.

  In the field sequential type electronic endoscope apparatus, red (R), green (G), and blue (B), for example, have different color wavelength ranges on the optical path of illumination light emitted from the lamp in the light source apparatus. A rotating filter having an opening in which three color transmission filters that transmit light are arranged and a light shielding part other than the opening is inserted. By rotating the rotating filter, illumination light is transmitted. The subject is interrupted intermittently, and each color light is sequentially irradiated onto the subject. Then, an image of the subject obtained for each color is picked up in time series by the CCD arranged at the tip of the electronic endoscope.

  At this time, the CCD performs exposure and light shielding in accordance with the rotation timing of the rotary filter. During the exposure period, new charges are stored in all the pixels of the CCD, and during the light blocking period, which is when the illumination light is blocked, all the stored charges are read out. This charge accumulation and readout operation is repeated, and an image signal generated by imaging with the CCD is input to the image signal processing unit.

  In general, these moving image signals include noise components, and frame recursive noise reduction processing is known as a technique for reducing the noise components. Here, the frame cyclic noise reduction processing is to detect the motion of the image signal from the output image one frame before and the current input image, and to control the degree of time average according to the amount of motion.

The performance of the frame recursive noise reduction device greatly depends on the accuracy of motion detection described above. Here, as a general motion detection algorithm, it is known to calculate a difference absolute value of luminance of an image frame signal and compare the difference absolute value with a motion determination threshold value. Patent Document 1 discloses a method of determining that a motion is absolute when the difference absolute value is larger than a motion determination threshold, and determining that the motion is stationary when the absolute value is smaller.
JP 2005-150903 A

  However, as described above, in the field sequential imaging method, for example, by rotating a rotation filter in which three color transmission filters of R, G, and B are arranged, an image of a subject for each color is time-sequentially in the CCD. I'm shooting. For this reason, if one or both of the subject and the imaging side move during shooting, a position shift of the field signal of each color occurs in the frame signal. For this reason, there is a problem that the spatial frequency of luminance in the moving image signal is lowered, the accuracy of motion detection is lowered, and efficient noise reduction cannot be performed.

  The present invention has been made in view of the above problems, and an image processing apparatus that performs an effective noise reduction process in consideration of the result of motion detection in an area sequential imaging method, and an endoscope apparatus including the image processing apparatus The purpose is to provide.

In order to solve the above problems, the present invention employs the following means.
The image processing apparatus according to the present invention includes a noise reduction unit that generates a second image field signal of each color by performing noise reduction processing on the first image field signal of each color sequentially input by the frame sequential method, and each color. Image storage means for storing the second image field signal, the first image field signal, and the first image created by noise reduction processing for the first image field signal in the past and stored in the image storage means Using the second image field signal of the same color as the field signal, a motion detection means for detecting the motion of the subject, and using the motion of the subject and the past second image field signal, A predicted image creating means for creating a predicted image field signal of the same color, and the noise reducing means is the processing target The noise reduction process is performed on the first image field signal input at the time to be processed by synthesizing the first image field signal input in step 1 and the predicted image field signal. .

  According to the image processing apparatus of the present invention, for each color, the movement of the subject is detected using the first image field signal and the second image field signal of the same color as the stored first image field signal. A predicted image field signal is generated using the movement of the subject and the past second image field signal of the same color as the first image field signal. Then, by performing image synthesis of the first image field signal that is the current image and the predicted image field signal of the same color in consideration of the movement of the subject, the first image field signal of each color captured by the frame sequential method is obtained. It becomes possible to reduce noise.

  In the image processing apparatus according to the present invention, motion storage means for storing subject motion information detected by the motion detection means, and motion information for a color to be processed are stored in the motion storage means. Motion correction means for correcting using the motion information of another color.

  For example, in an endoscope that captures the inside of a body cavity, an image signal in a blue (B) field has a lower signal level and a lower contrast than an image signal in a red (R) or green (G) field. Information detection accuracy is lowered. As a result, there is a problem that noise cannot be effectively reduced because the predicted image signal of the B field cannot be generated with high accuracy. According to the image processing apparatus of the present invention, it is possible to correct the motion information of the image signal of the B field using the motion information of the image signal of the R or G field. Thereby, the predicted image signal of the B field can be generated with high accuracy, and the noise reduction effect can be improved.

  The motion correction unit corrects the motion information of a color having a low spatial frequency band using the motion information of the subject having a color having a high spatial frequency band.

For colors with a low spatial frequency band, the accuracy of the motion information of the subject is low.
According to the image processing apparatus of the present invention, a subject having a color with a low spatial frequency band is corrected by correcting the motion of the subject with a color having a low spatial frequency band using the motion information of the subject having a color with a high spatial frequency band. The motion detection can be performed with high accuracy.

  The image processing apparatus according to the present invention includes a synthesis ratio determining unit that determines a synthesis ratio between the first image field signal and the predicted image field signal in accordance with the movement of the subject, and the synthesis ratio determining unit The first image field signal is subjected to noise reduction processing by combining the first image field signal and the predicted image field signal with the determined combination ratio.

  According to the image processing apparatus of the present invention, the composition ratio between the first image field signal that is the current image and the predicted image field signal is determined based on the motion of the subject, and image composition is performed according to the composition ratio. This makes it possible to perform noise reduction processing according to the movement of the subject.

  The composition ratio determining means sets the composition ratio so that the ratio of the first image field signal is increased in an area where the subject motion is large, and the ratio of the predicted image field signal is increased in an area where the subject motion is small. It is characterized by determining.

  According to the image processing apparatus of the present invention, it is possible to effectively reduce noise by increasing the ratio of the predicted image field signal in a portion where the movement of the subject is small. On the other hand, with respect to a portion where the movement of the subject is large, the occurrence of an afterimage can be suppressed by increasing the ratio of the first image field signal that is the current image.

  The image processing apparatus according to the present invention includes an image error calculation means for calculating an image error between the first image field signal and the predicted image field signal, and the first image field signal according to the image error. Combining ratio determining means for determining a combining ratio with the predicted image field signal, and combining the first image field signal and the predicted image field signal with the combining ratio determined by the combining ratio determining means. Thus, noise reduction processing is performed on the first image field signal.

  According to the image processing apparatus of the present invention, the composition ratio is determined based on the image error calculated by the image error calculation means, and the image composition is performed according to this composition ratio. This makes it possible to perform noise reduction processing according to the image error.

  In the image processing apparatus according to the present invention, the synthesis ratio determining means increases the ratio of the first image field signal in a region where the image error is large, and sets the ratio of the predicted image field signal in a region where the image error is small. The composition ratio is determined so as to increase.

  According to the image processing apparatus of the present invention, it is possible to effectively reduce noise by increasing the ratio of the predicted image field signal in a portion where the image error is small. On the other hand, in the portion where the image error is large, the occurrence of afterimage can be suppressed by increasing the ratio of the first image field signal which is the current image.

  An endoscope apparatus according to the present invention includes any one of the image processing apparatuses described above.

  According to the endoscope apparatus according to the present invention, noise in the image signal of each field can be reduced, so that it is possible to observe a subtle color change of the subject.

  According to the present invention, the first image field signal captured by the frame sequential method is obtained by performing image synthesis of the first image field signal that is the current image and the predicted image field signal of the same color in consideration of the movement of the subject. On the other hand, it is possible to reduce a suitable noise.

[First Embodiment]
The image processing apparatus according to the first embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a schematic configuration diagram of an endoscope apparatus to which an image processing apparatus according to the present invention is applied.
As shown in FIG. 1, an endoscope apparatus according to the present invention includes an electronic endoscope 11 including an image sensor, an image processing device 12 that performs signal processing on an output signal of the image sensor, and the electronic endoscope 11. A light source device 13 that supplies illumination light to the monitor, and a monitor 14 that receives an image signal from the image processing device 12 and displays a captured endoscopic image.

The main body of the electronic endoscope 11 includes an operation unit 15, an elongated insertion unit 16 provided on the distal end side of the operation unit 15, and a side opposite to the operation unit 15 connected to the side of the operation unit 15. And a branched cable 17.
Connectors 18 and 19 are provided at the branch ends of the cable 17, respectively. The cable 17 is connected to the image processing device 12 via a connector 18 and to the light source device 13 via a connector 19.
An objective lens 20 and a CCD 21 as a solid-state imaging device are attached to the distal end of the insertion portion 16, and the CCD 21 is disposed at the image forming position of the objective lens 20. A drive signal line 22 and an output signal line 23 are connected to the CCD 21, and the drive signal line 22 and the output signal line 23 are connected to the image processing apparatus 12 via a connector 18.
The electronic endoscope 11 is provided with a light guide 24 that is a glass fiber bundle that transmits illumination light to the distal end of the insertion portion 16, and the light guide 24 is connected to the light source device 13 via the connector 19. ing. Thereby, the illumination light emitted from the light source device 13 enters the incident end of the light guide 24.

The image processing device 12 is for performing frame-sequential imaging, noise reduction processing, and synchronization processing, and includes a preprocessing circuit 30, a CCD drive circuit 31, a control circuit 32, and an A / D converter 33. , A noise reduction circuit 34, a synchronization circuit 35, a synchronization circuit 35, a D / A converter 36, and a post-processing circuit 37.
The preprocessing circuit 30 and the CCD drive circuit 31 are connected to the CCD 21 via the connector 18. The preprocessing circuit 30 is connected in the order of an A / D converter 33, a noise reduction circuit 34, a synchronization circuit 35, a synchronization circuit 35, a D / A converter 36, a post-processing circuit 37, and the monitor 14. Further, a control line is connected to each circuit from the control circuit 32 to control a generated timing signal and supply the signal to each circuit unit.

  The light source device 13 includes a light source lamp 25 that generates white light, a motor 26, a rotary filter 27 that is driven by the motor 26, and has color transmission filters of three primary colors of R, G, and B, a condenser lens 28, and a motor. And a drive circuit 29 for controlling the H.26.

The flow of image signals in the endoscope apparatus having the above configuration will be described below.
The white light emitted from the light source lamp 25 passes through the rotary filter 27 arranged in the optical path, and is sequentially made into illumination light of each wavelength of R, G, B, and then collected by the condenser lens 28. Light is incident on the incident end of the light guide 24. The light reflected from the subject by the R, G, and B illumination lights emitted from the tip of the light guide 24 passes through the objective lens 20 and forms an image on the imaging surface of the CCD 21, and is photoelectrically converted and captured from the optical image. Converted to a signal. The rotary filter 27 is driven by a motor 26 controlled by a drive circuit 29 and normally rotates at 20 Hz. Further, exposure and light shielding by the CCD 21 are controlled to be performed in accordance with the rotation of the rotary filter 27. For example, as described above, when the rotary filter 27 rotates at 20 Hz, 1/60 Hz for each color. The exposure and the light shielding are repeated at the timing.

In the image processing apparatus 12, superimposition of a character / image mask signal and switching of a menu screen / test screen (color bar, etc.) are performed by an operation from an input means such as a keyboard (not shown) connected to the control circuit 32.
The CCD drive circuit 31 generates a drive signal for driving the CCD 21 based on a signal from the control circuit 32. The image signal from the CCD 21 driven in this way is input to the preprocessing circuit 30. The image signal input to the preprocessing circuit 30 is subjected to predetermined processing such as amplification and waveform shaping, and then converted into digital data by the A / D converter 33. The image signal converted into digital data is subjected to noise reduction processing by the noise reduction circuit 34 and then input to the synchronization circuit 35. The synchronization circuit 35 performs a frame sequential signal synchronization process and inputs the signal to the D / A converter 36. The image signal input to the D / A converter 36 is converted into an analog signal. The analog image signal is input to the post-processing circuit 37 and subjected to predetermined processing such as gain adjustment, and then input to the monitor 14. The monitor 14 displays an image of the subject based on the image signal subjected to the above processing.

Details of the processing in the noise reduction circuit 34 in the above processing will be described below.
FIG. 2 is a functional block diagram showing the functions provided in the noise reduction circuit 34 in an expanded manner.
The noise reduction circuit 34 includes an image input unit 41, an image storage unit (image storage unit) 42, a selector 52, a motion detection unit (motion detection unit) 43, a motion storage unit (motion storage unit) 44, a motion A correction unit (motion correction unit) 45, a predicted image generation unit (prediction image generation unit) 46, a noise reduction unit (noise reduction unit) 47, an image composition ratio determination unit (composition ratio determination unit) 48, and a selector 51 And an image output unit 49.

The flow of the image signal in the noise reduction circuit 34 having the above configuration will be described below.
R, G, and B field signals converted into digital data by the A / D converter 33 are sequentially input to the image input unit 41 sequentially at predetermined time intervals.

The image storage unit 42 accumulates noise-reduced image field signals (second image field signals) output from a noise reduction unit 47 described later for each color. Specifically, the image storage unit 42 has R, G, and B field signal memories, and the second image field signal output from the noise reduction unit 47 is switched by switching the selector 51 so that each field signal is output. Accumulate in memory according to color.
Further, the second image field signal accumulated in the image storage unit 42 switches the selector 52 so that the field signal of the same color as the current image field signal input from the image input unit 41 is converted into the motion detection unit 43 and the predicted image. The data is output to the creation unit 46.

  The motion detector 43 detects the motion of the subject between the current image field signal and the second image field signal of the same color stored in the image storage unit 42. Specifically, first, R, G, and B field signals converted into digital data are sequentially input to the image input unit 41. The input image field signal is read into the motion detector 43 as a current image field signal. In addition, at the timing when the current image field signal is read by the motion detection unit 43, the second image field signal one frame before the current image field signal having the same color as the current image field signal stored in the image storage unit 42 is selected by the selector 52. Is read by the motion detection unit 43. The movement of the subject is calculated by comparing the second image field signal of the previous frame stored in the image storage unit 42 with the current image field signal.

  For the motion detection, for example, a method is used in which the current image is divided into blocks and a motion from an image (reference image) one frame before is detected for each block. In this method, a position with the smallest difference from the target block is specified within the search range of the reference image with respect to the target block in the divided blocks of the current image. Based on the specified position, a vector (hereinafter referred to as “motion vector”) representing the moving direction and moving amount of the subject in units of the block with respect to the block of interest is determined. In other words, the absolute difference value in block units or the similarity evaluation value corresponding to the absolute difference value is calculated for all vectors within the search range, and the target block is calculated based on the calculated absolute difference value or similarity evaluation value. Is determined, and the motion vector of the block of interest is determined.

As the block size, for example, sizes of 16 × 16, 16 × 8, 8 × 16, and 8 × 8 pixels are defined. In addition, when the motion detection unit 43 finishes detecting the motion of the subject between the second image field signal one frame before and the current image field signal in all blocks by the motion detection method described above, the motion detection unit 43 detects the motion of the subject. Presence / absence, amount, direction, and the like are output to the motion storage unit 44 and the motion correction unit 45 as subject motion information.
The motion storage unit 44 accumulates the subject motion information from the motion detection unit 43.

  The motion correction unit 45 corrects the motion information of the subject detected by the motion detection unit 43 using the motion information in the image field signal of another color accumulated in the motion storage unit 44 according to the accuracy of motion detection. I do. The motion information input to the motion correction unit 45 calculates the accuracy of motion detection for each block of the image.

  The step of calculating the accuracy of motion detection is performed, for example, by comparing the motion information of the block of interest with the motion information in blocks near 4 or 8. Specifically, the motion detection accuracy is calculated using a correlation value based on the motion vector of the block of interest and the magnitude and angle of the motion vector of each neighboring block. Or you may use the spatial frequency band of the search range which detected the motion information of the attention block. Further, both the correlation value and the spatial frequency band described above may be used. Here, when the spatial frequency of the search range is low, the image motion detection accuracy is low, so the detection accuracy is calculated using the spatial frequency in the search range of the reference image.

Next, the motion information of the image field signal of another color accumulated in the motion storage unit 44 is read out according to the accuracy of motion detection thus obtained, and the motion information is corrected. For example, when the accuracy of motion detection is lower than the accuracy determination threshold, the motion information is corrected by replacing the motion information at the same position in the field signal of another color with high detection accuracy accumulated in the motion storage unit 44. Do. Alternatively, a value extrapolated from the motion information of the two image field signals accumulated in the motion storage unit 44 may be used. In particular, in an endoscope that captures the inside of a body cavity, since the R and G fields have a higher signal level and a higher spatial frequency band than the B field, motion detection can be performed with high accuracy. The motion information of the B field may be corrected using the motion information of the R and G fields stored in the.
In this way, the motion information corrected by the motion correction unit 45 is output to the predicted image creation unit 46 and the image composition ratio determination unit 48.

  The predicted image creation unit 46 creates a predicted image field signal based on the second image field signal having the same color as the current image read from the image storage unit 42 and the motion information provided by the motion correction unit 45. Specifically, the second image field signal read from the image storage unit 42 is used as a reference image, and predicted image data is created from a block of the reference image at a position shifted by the motion information. Further, the predicted image creation unit 46 outputs the predicted image field signal to the noise reduction unit 47.

  The image composition ratio determining unit 48 determines a composition ratio between the current image field signal and the predicted image field signal based on the motion information of the subject. Specifically, the image composition ratio determination unit 48 predicts the pixel of the part (the still part and the part close to the still part) in which the amount of motion is small in the current image field signal based on the input motion information for each pixel. The composition ratio is determined so that the ratio of pixel values in the pixels of the image field signal is high. On the other hand, for the pixels of the portion where the amount of motion in the current image field signal is large (the motion portion and the portion close to motion), the composition ratio is determined so that the ratio of the pixel values of the pixels in the current image is high. For example, when the amount of movement of the subject between the predicted image and the current image is large, the ratio of the current image is 1 or a value close to 1.

  Even with the same image composition ratio, the image quality differs depending on factors such as the characteristics of the imaging device and the characteristics of the motion information, so the operator should actually look at the image and adjust the image composition ratio. May be. For example, the image composition ratio determination unit 48 sets the noise reduction intensity by the operator and adjusts the composition ratio according to the noise reduction intensity. When the noise reduction intensity is set strongly, the synthesis ratio is adjusted so that the ratio of the pixel values in the pixels of the predicted image field signal is high. On the other hand, when the noise reduction intensity is set low, the composition ratio is adjusted so that the ratio of the pixel values of the pixels in the current image is high. Further, the image composition ratio determination unit 48 outputs the image composition ratio determined in this way to the noise reduction unit 47.

  The noise reduction unit 47 performs noise reduction processing on the current image field signal using the synthesis ratio information provided from the image synthesis ratio determination unit 48. Specifically, the noise reduction unit 47 inputs the image composition ratio from the image composition ratio determination unit 48, the current image field signal from the image input unit 41, and the predicted image field signal from the predicted image creation unit 46. , And the noise between fields is reduced for the current image field signal based on the image composition ratio. The inter-field noise reduction method uses the pixel value of a pixel having the same in-field position in each of the current image field signal and the predicted image field signal, and an image composition ratio corresponding to the pixel as follows ( The pixel value of the noise-reduced image is calculated by the equation 1).

The combined image field signal is output to the image storage unit 42 and the image output unit 49 as a second image field signal after noise reduction.
The second image field signal stored in the image storage unit 42 is used as an image field signal for the previous frame and for noise reduction of the subsequent first image field signal. The image output unit 49 sequentially outputs the noise-reduced second image field signal output from the noise reduction unit 47 to the synchronization circuit 35.

  In the above embodiment, if an intra-field noise reduction unit is arranged between the image input unit 41 and the noise reduction unit 47 to reduce noise in the current image field signal, the image composition ratio is increased. Even when the composition ratio of the current image is set large in the determining unit 48, an image having a noise reduction effect can be obtained. In-field noise reduction methods include, for example, a method of smoothing singular points using a median filter, a method of smoothing random noise while preserving edge components using a nonlinear filter such as a bilateral filter, and shooting in a dark state. There is a method of subtracting the fixed pattern noise output from the element for each pixel. Thus, the image field signal with reduced noise is output to the noise reduction unit 47 in accordance with the in-field noise reduction technique.

  As described above, according to the image processing apparatus of the present embodiment, the movement of the subject is detected using the first image field signal and the stored second image field signal, and the movement of the subject and the second image field are detected. The predicted image field signal is generated using the signal. Here, since the second image field signal is a signal having the same color in consideration of the motion of the subject, image synthesis is performed between the first image field signal that is the current image and the predicted image field signal of the same color in consideration of the motion of the subject. Thus, it is possible to perform suitable noise reduction on the first image field signal imaged by the frame sequential method.

  Further, for example, in an endoscope that captures the inside of a body cavity, an image signal in a blue (B) field has a lower signal level and a lower contrast than an image signal in a red (R) or green (G) field. , Motion information detection accuracy is lowered. As a result, there is a problem that noise cannot be effectively reduced because the predicted image signal of the B field cannot be generated with high accuracy. The image processing apparatus according to the present embodiment can correct the motion information of the B field image signal using the motion information of the R and G field image signals. Thereby, the predicted image signal of the B field can be generated with high accuracy, and the noise reduction effect can be improved.

In addition, for the color having a low spatial frequency band, the accuracy of the motion information of the subject is low.
According to the image processing apparatus of the present invention, a subject having a color with a low spatial frequency band is corrected by correcting the motion of the subject with a color having a low spatial frequency band using the motion information of the subject having a color with a high spatial frequency band. The motion detection can be performed with high accuracy.

  Further, based on the movement of the subject, a synthesis ratio between the first image field signal that is the current image and the predicted image field signal is determined, and image synthesis is performed in accordance with the synthesis ratio. This makes it possible to perform noise reduction processing according to the movement of the subject.

Furthermore, noise can be effectively reduced by increasing the ratio of the predicted image field signal in a portion where the movement of the subject is small. On the other hand, with respect to a portion where the movement of the subject is large, the occurrence of an afterimage can be suppressed by increasing the ratio of the first image field signal that is the current image.
It is also possible to set the composition ratio according to shooting conditions and user settings.

  Further, according to the endoscope apparatus according to the present embodiment, since the image signal noise in each field can be reduced by providing the above-described image processing apparatus, a subtle color change of the subject is observed. It becomes possible.

  In the above embodiment, processing by hardware is assumed, but it is not necessary to be limited to such a configuration. For example, a configuration is possible in which the signal from the CCD 21 is output as raw data without processing, the information at the time of photographing from the control circuit 32 is output as header information, and separately processed by software. In this case, the image processing apparatus includes a main storage device such as a CPU and a RAM, and a computer-readable recording medium on which a program for realizing all or part of the above processing is recorded. The CPU reads out the program recorded in the storage medium and executes information processing / arithmetic processing, thereby realizing processing similar to that of the above-described image processing apparatus. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like.

When the noise reduction processing in the present embodiment is performed by software, first, the entire processing will be described below with reference to FIG.
In Step 1, the first image field signal and the second image field signal are read. In Step 2, image motion information is detected from each read image field signal. In Step 3, the detected motion information is corrected using motion information of another color. In Step 4, a predicted image is created from the image field signal (second image field signal) one frame before. In Step 5, the image synthesis ratio between the first image field signal and the predicted image field signal is determined based on the motion information of the subject. In Step 6, the inter-field noise of the first image field signal is reduced by synthesizing the first image field signal and the predicted image field signal. In Step 7, the noise-reduced image (second image field signal) is stored in the image storage unit. At Step 8, the noise-reduced image (second image field signal) is output and the process ends.
Next, detailed processing in each of the above steps will be described below.

FIG. 4 is a flow relating to the motion detection process in Step 2 described above.
In Step 11, the first image field signal is read as the current image field signal. At Step 12, the second image field signal of the previous frame having the same color as the current image field signal stored in the image storage unit is read. In Step 13, the current image field signal is divided into a plurality of blocks, and the block of interest is extracted. In Step 14, the similarity evaluation value with this block of interest is calculated within the search range of the second image field signal one frame before. In Step 15, it is determined whether the calculation of the similarity evaluation value is completed in the entire search range of the second image field signal one frame before. If not completed, the process branches to Step 14, and if completed, the process branches to Step 16. To do. In Step 16, it is determined whether the image motion detection has been completed in all blocks of the current image. If not completed, the process branches to Step 13, and if completed, the process branches to Step 17. In Step 17, the motion information of the detected image is stored in the motion storage unit. At Step 18, the motion information of the detected image is output and the process ends.

FIG. 5 is a flow relating to the motion correction process in Step 3 described above.
At Step 21, the motion information of the current image field signal is read. In Step 22, the accuracy of motion detection is calculated for each block of the image. In Step 23, it is determined whether or not the accuracy of motion detection is smaller than the determination threshold value. If the detection accuracy is smaller than the threshold value, the process branches to Step 24. If the detection accuracy is larger than the threshold value, the process branches to Step 26. In Step 24, the motion information of the image field signal of another color accumulated in the motion storage unit is read out. In Step 25, the motion information of the current image field signal is corrected based on the motion information of the image field signal of another color. At Step 26, the corrected subject movement information is output and the process ends.

FIG. 6 is a flow relating to the predicted image creation processing in Step 4 described above.
At Step 31, the second image field signal is read one frame before the same color as the current image field signal. In Step 32, the corrected movement information of the subject is read out. In Step 33, a predicted image field signal is created by shifting the second image field signal of the previous frame based on the motion information of the subject. At Step 34, the generated predicted image field signal is output and the process ends.

FIG. 7 is a flow relating to the image composition ratio determination process in Step 5 described above.
In Step 41, the motion information of the current image field signal is read. At Step 42, for the pixels in the portion where the amount of motion in the current image field signal is small, the portion in which the amount of motion in the current image field signal is large while the ratio of the pixel values in the pixels of the predicted image field signal is high For these pixels, the composition ratio is determined so that the ratio of the pixel values of the pixels in the current image is high. At Step 43, the image composition ratio determined in this way is output and the process ends.

FIG. 8 is a flow relating to the noise reduction processing in Step 6 described above.
In Step 51, the image composition ratio is read. At Step 52, the current image field signal is read. In Step 53, the predicted image field signal is read. In Step 54, the pixel value of the noise-reduced image is obtained using the pixel value of the pixel having the same position in the field in each of the current image field signal and the predicted image field signal and the image composition ratio corresponding to the pixel. Is calculated.

  In Step 7, the image field signal combined in this way is output as a second image field signal after noise reduction, and the process ends.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG.
The image processing apparatus according to the present embodiment is different from the first embodiment in that the synthesis ratio between the first image field signal and the predicted image field signal is determined by the image error of each image field signal. Hereinafter, the image processing apparatus according to the present embodiment will not be described with respect to the points common to the first embodiment, and different points will be mainly described.

  FIG. 9 shows a configuration in which an image error calculation unit (image error calculation means) 50 for calculating an image error from the current image field signal and the predicted image field signal is added in the first embodiment described above. The basic configuration is the same as that of the first embodiment, and the same name and number are assigned to the same configuration.

  The motion information of the image corrected by the motion correction unit 45 is output to the predicted image creation unit 46. The predicted image field signal created by the predicted image unit 46 is output to the noise reduction unit 47 and the image error calculation unit 50.

  The image error calculation unit 50 calculates an image error between the current image field signal and the predicted image field signal. Specifically, for the current image field signal input from the image input unit 41 and the predicted image field signal input from the predicted image creation unit 46, each image is divided into a plurality of blocks. First, a target block is extracted, and an image error value is calculated for each block using pixel values in blocks having the same position in the field. The image error value is determined by an absolute difference value in block units, a square difference value, or an evaluation value corresponding thereto. As a corresponding evaluation value, for example, in SNR (Signal to Noise Ratio), the signal is the maximum value of the image, and noise is the error between the predicted image and the current image. Ratio) may be used.

Further, a lookup table may be used for calculating the image error value. Further, the image error calculation unit 50 outputs the image error value determined in this way to the image composition ratio determination unit 48.
The image composition ratio determination unit 48 determines a composition ratio between the current image field signal and the predicted image field signal based on the image error value provided from the image error calculation unit 50. Specifically, in the image composition ratio determination unit 48, based on the input image error value for each block, the pixel value in the pixel of the predicted image field signal is the pixel of the portion where the image error amount is small in the current image field signal. The synthesis ratio is determined so that the ratio of is high. On the other hand, the composition ratio is determined so that the ratio of the pixel value of the pixel in the current image is high for the pixel in the portion where the amount of image error in the current image field signal is large. For example, when the amount of image error between the predicted image and the current image is large, the ratio of the current image is 1 or a value close to 1.

  In the present embodiment, if the intra-field noise reduction unit is arranged in front of the noise reduction unit 47 so as to reduce noise in the current image field signal, the image composition ratio determination unit 48 can reduce the current image. Even when the composite ratio is set large, an image having a noise reduction effect can be obtained. The image field signal with reduced intra-field noise is output to the noise reduction unit 47.

  As described above, according to the image processing apparatus according to the present embodiment, the composition ratio is determined based on the image error calculated by the image error calculation unit 50, and image composition is performed according to the composition ratio. This makes it possible to perform noise reduction processing according to the image error.

In addition, in a portion where the image error is small, it is possible to effectively reduce noise by increasing the ratio of the predicted image field signal. On the other hand, in the portion where the image error is large, the occurrence of afterimage can be suppressed by increasing the ratio of the first image field signal which is the current image.
It is also possible to set the composition ratio according to shooting conditions and user settings.

  In the above embodiment, processing by hardware is assumed, but it is not necessary to be limited to such a configuration. For example, a configuration is possible in which the signal from the CCD 21 is output as raw data without processing, the information at the time of photographing from the control circuit 32 is output as header information, and separately processed by software.

When the noise reduction process in the present embodiment is performed by software, first, the entire process will be described below with reference to FIG. Note that the same number of steps is assigned to the same processing step as the software flow of the noise reduction processing in the first embodiment shown in FIG.
At Step 1, an image field signal is read. In Step 2, image motion information is detected. In Step 3, the detected motion information is corrected using motion information of another color. In Step 4, a predicted image is created from the image field signal of the previous frame. In Step 61, an image error value between the current image field signal and the predicted image field signal is calculated. In Step 62, the image composition ratio is determined based on the image error value. At Step 6, noise between fields is reduced. In Step 7, the noise-reduced image is accumulated in the image storage unit. In Step 8, the noise-reduced image is output and the process ends.
Next, detailed processing in each of the above steps will be described below.

FIG. 11 is a flow relating to the motion detection process in Step 2 described above.
In Step 11, the first image field signal is read as the current image field signal. At Step 12, the second image field signal of the previous frame having the same color as the current image field signal stored in the image storage unit is read. In Step 13, the current image field signal is divided into a plurality of blocks, and the block of interest is extracted. In Step 14, the similarity evaluation value with this block of interest is calculated within the search range of the second image field signal one frame before. In Step 15, it is determined whether the calculation of the similarity evaluation value is completed in the entire search range of the image field signal one frame before. If not completed, the process branches to Step 14, and if completed, the process branches to Step 16. In Step 16, it is determined whether the image motion detection has been completed in all blocks of the current image. If not completed, the process branches to Step 13, and if completed, the process branches to Step 17. In Step 17, the motion information of the detected image is stored in the motion storage unit. At Step 18, the motion information of the detected image is output and the process ends.

FIG. 12 is a flow related to the motion correction process in Step 3 described above.
At Step 21, the motion information of the current image field signal is read. In Step 22, the accuracy of motion detection is calculated for each block of the image. In Step 23, it is determined whether or not the accuracy of motion detection is smaller than the determination threshold value. If the detection accuracy is smaller than the threshold value, the process branches to Step 24. If the detection accuracy is larger than the threshold value, the process branches to Step 26. In Step 24, the motion information of the image field signal of another color accumulated in the motion storage unit is read out. In Step 25, the motion information of the current image field signal is corrected based on the motion information of the image field signal of another color. In Step 26, the motion information of the corrected image is output and the process ends.

FIG. 13 is a flow relating to the predicted image creation processing in Step 4 described above.
At Step 31, the second image field signal of the previous frame having the same color as the current image field signal is read out. At Step 32, the corrected image motion information is read out. In Step 33, an image obtained by shifting the second image field signal of the previous frame is generated based on the motion information of the image, and is set as a predicted image field signal. At Step 34, the generated predicted image field signal is output and the process ends.

FIG. 14 is a flow relating to the image error calculation process in Step 61.
At Step 71, the current image field signal is read. In Step 72, the predicted image field signal is read. In Step 73, the current image field signal is divided into a plurality of blocks, and the block of interest is extracted. In Step 74, an image error value is calculated between the predicted image field signal block and the same position as the target block in the current image field signal. In Step 75, it is determined whether the calculation of the image error value is completed in all blocks of the current image field signal. If not completed, the process branches to Step 73. If completed, the process branches to Step 76. At Step 76, an image error value is output and the process ends.

FIG. 15 is a flow relating to the image composition ratio determination process in Step 62 described above.
In Step 81, an image error value between the current image field signal and the predicted image field signal is read. In Step 82, the ratio of the pixel value in the pixel of the predicted image field signal is increased for the pixel in the portion having a small image error value, while the pixel in the current image is set in the portion in which the image error value is large. The composition ratio is determined so that the ratio of the pixel values becomes high. At Step 43, the image composition ratio determined in this way is output and the process ends.

FIG. 16 is a flow relating to the noise reduction processing in Step 6 described above.
In Step 51, the image composition ratio is read. At Step 52, the current image field signal is read. In Step 53, the predicted image field signal is read. In Step 54, the pixel value of the noise-reduced image is obtained using the pixel value of the pixel having the same position in the field in each of the current image field signal and the predicted image field signal and the image composition ratio corresponding to the pixel. Is calculated.

  At Step 7, the combined image field signal is output as a second image field signal having been subjected to inter-field noise reduction, and the process ends.

1 is a schematic configuration diagram of an endoscope apparatus according to the present invention. FIG. 3 is a functional block diagram showing expanded functions provided in the noise reduction circuit of the image processing apparatus according to the first embodiment. It is a flowchart of the noise reduction process which concerns on 1st Embodiment. It is a flowchart of the motion detection process in the process shown in FIG. It is a flowchart of the motion correction process in the process shown in FIG. It is a flowchart of the prediction image creation process in the process shown in FIG. It is a flowchart of the image composition ratio determination process in the process shown in FIG. It is a flowchart of the noise reduction process in the process shown in FIG. It is the functional block diagram which expand | deployed and showed the function with which the noise reduction circuit of the image processing apparatus which concerns on 2nd Embodiment is provided. It is a flowchart of the noise reduction process which concerns on 2nd Embodiment. It is a flowchart of the motion detection process in the process shown in FIG. It is a flowchart of the motion correction process in the process shown in FIG. It is a flowchart of the estimated image creation process in the process shown in FIG. It is a flowchart of the image error calculation process in the process shown in FIG. It is a flowchart of the image composition ratio determination process in the process shown in FIG. It is a flowchart of the noise reduction process in the process shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Endoscopic apparatus 12 Image processing apparatus 42 Image storage part 43 Motion detection part 44 Motion storage part 45 Motion correction part 46 Predictive image creation part 47 Noise reduction part 48 Composition ratio determination part 50 Image error calculation part

Claims (8)

Noise reduction means for generating a second image field signal of each color by performing noise reduction processing on the first image field signal of each color sequentially input by a frame sequential method;
Image storage means for storing the second image field signal of each color;
Using the first image field signal and the second image field signal having the same color as the first image field signal, which is created by noise reduction processing for the past first image field signal and stored in the image storage means Motion detection means for detecting the motion of the subject;
Predicted image creating means for creating a predicted image field signal of the same color as the first image field signal using the movement of the subject and the second image field signal in the past;
With
The noise reduction unit synthesizes the first image field signal input at the time of the processing target and the predicted image field signal, and outputs the first image field signal input at the time of the processing target. An image processing apparatus that performs the noise reduction process. Movement storage means for storing movement information of the subject detected by the movement detection means;
Motion correction means for correcting the motion information about the color to be processed using the motion information of another color stored in the motion storage means;
The image processing apparatus according to claim 1, comprising:   The image processing apparatus according to claim 2, wherein the motion correction unit corrects the motion information of a color having a low spatial frequency band using the motion information of a color having a high spatial frequency band. A synthesis ratio determining unit that determines a synthesis ratio of the first image field signal and the predicted image field signal according to the movement of the subject;
4. The noise reduction process for the first image field signal is performed by synthesizing the first image field signal and the predicted image field signal at a synthesis ratio determined by the synthesis ratio determination unit. An image processing apparatus according to claim 1.   The composition ratio determining means sets the composition ratio so that the ratio of the first image field signal is increased in an area where the subject motion is large, and the ratio of the predicted image field signal is increased in an area where the subject motion is small. The image processing apparatus according to claim 4, wherein the image processing apparatus is determined. Image error calculating means for calculating an image error between the first image field signal and the predicted image field signal;
Synthesis ratio determining means for determining a synthesis ratio of the first image field signal and the predicted image field signal in accordance with the image error;
Have
4. The noise reduction process for the first image field signal is performed by synthesizing the first image field signal and the predicted image field signal at a synthesis ratio determined by the synthesis ratio determination unit. An image processing apparatus according to claim 1.   The composition ratio determining means determines the composition ratio so that the ratio of the first image field signal is increased in a region where the image error is large and the ratio of the predicted image field signal is increased in a region where the image error is small. The image processing apparatus according to claim 6.   An endoscope apparatus comprising the image processing apparatus according to claim 1.

Images