JPWO2010150681A1 - Image input device - Google Patents

Abstract

An object of the present invention is to provide an image input device that can remove noise from a moving image while preventing edge blur while having a simple configuration. A change in the subject between frames that temporally precedes the frame is detected for each predetermined region, and is classified into a plurality of states according to the change in the subject. At least one of the chromaticity signals is averaged for each of the predetermined areas using at least one of the luminance signal and the chromaticity signal corresponding to a predetermined number of the preceding frames, and image data forming means Is combined with the luminance signal and the chromaticity signal processed by the processing means to form image data corresponding to the processing target frame

Description

  The present invention relates to an image input apparatus provided with an imaging means such as an image sensor, and more particularly to an image input apparatus suitable for capturing a moving image.

  A plurality of still images are continuously photographed to create a plurality of frames, which are switched at predetermined time intervals and displayed, thereby obtaining a moving image showing the movement of the subject. By the way, the imaging conditions at the time of imaging are not always constant, and in addition, noise is often generated in the process of forming a frame due to the influence of the dark current of the imaging device, image processing, and the like. As a measure for removing noise from the frame, a method of obtaining an average between a plurality of frames is used, but there is a problem that an edge portion that moves quickly in the time direction is blurred.

  On the other hand, in Patent Document 1, each frame is analyzed by a plurality of filters having different resolution scales, and a moving average between a number of frames set according to the resolution of each analysis image for each resolution scale. In the moving average means, the number of frames for which the moving average is obtained is determined so that the moving average is obtained among a larger number of frames as the image is analyzed on the lower resolution scale, and each resolution scale obtained by the moving average means is obtained. A technique is disclosed in which noise is removed while blurring of edges is prevented by synthesizing these images and obtaining a moving image based on the original moving image sequence.

Japanese Patent Laid-Open No. 08-154188

  However, according to the prior art of Patent Document 1, each frame must be analyzed by a plurality of filters having different resolution scales, which takes time to process and increases costs by using expensive filters. There is a problem of inviting. In addition, the technique disclosed in Patent Document 1 has a problem that it cannot cope with a large change in the subject.

  The present invention has been made in view of the above-described problems, and an object thereof is to provide an image input apparatus that can remove noise from a moving image while preventing edge blurring with a simple configuration.

An image input device according to claim 1 is provided.
Imaging means for entering subject light and converting it into an image signal;
Separating means for separating the image signal sequentially output from the imaging means into a luminance signal and a chromaticity signal for each frame;
Changes in the subject between a processing target frame in a plurality of temporally continuous frames and a frame temporally preceding the processing target frame are detected for each predetermined region, and classified into a plurality of states according to the subject change Detecting means for
For at least one of the luminance signal and chromaticity signal in the processing target frame separated by the separation means, using at least one of the luminance signal and chromaticity signal corresponding to a predetermined number of the preceding frames, Processing means for performing different averaging processes according to the classification by the detection means for each predetermined region;
Image data forming means for combining the luminance signal and chromaticity signal processed by the processing means to form image data corresponding to the processing target frame.

  For example, when a moving image is captured of a subject that is almost stationary, the change in the subject between a plurality of frames is small, so by averaging the luminance signal and the chromaticity signal over relatively long frames, Noise can be removed. However, when a moving subject is imaged at high speed (including when the camera moves at high speed), the change in the subject between a plurality of frames becomes large. Therefore, the luminance signal and the chromaticity over a long-interval frame. When the signal is averaged, there is a high possibility that the blur of the edge becomes large.

  On the other hand, according to the present invention, the detection means detects a change in a subject between a processing target frame in a plurality of temporally continuous frames and a frame temporally preceding the processing target frame for each predetermined region. When detecting and classifying the plurality of states into, for example, a first state and a second state in which the subject change is larger than the first state, the first state and the first state are changed according to the subject change. 2 corresponding to at least one of the luminance signal and the chromaticity signal in the processing target frame separated by the separation unit and corresponding to the predetermined number of the preceding frames. Since the averaging process according to the classification by the detection unit is performed for each of the predetermined regions using at least one of the signal and the chromaticity signal, the detection unit is separated from the first state. Since it can be determined that the change in the subject is relatively small, for example, the noise can be reduced by increasing the number of frames to be averaged, while the change in the subject can be determined to be relatively large if classified as the second state. For example, the blurring of the edge can be effectively suppressed by changing the processing content of the processing means so that the number of frames to be averaged is reduced or zero. Therefore, it is possible to reduce noise while preventing edge blur with a simple configuration.

  When the averaging process is performed on the luminance signal in the processing target frame, the luminance signal in the temporally preceding frame is used, and when the averaging process is performed on the chromaticity signal in the processing target frame. Of course, the chromaticity signal in the temporally preceding frame is used.

  The image input device according to claim 2 is the image input device according to claim 1, wherein a difference between the luminance signal of the preceding frame and the luminance signal of the processing target frame is equal to or less than a first threshold value. When the detection unit classifies the change of the subject as the first state, the processing unit performs an averaging process using the luminance signal, and the luminance signal of the preceding frame, When the difference from the luminance signal of the processing target frame exceeds the first threshold, the detection unit classifies the change of the subject as the second state larger than the first state, The processing means performs an averaging process using the chromaticity signal. Human vision has a characteristic that image edge blur based on the luminance signal is relatively easy to detect, but image edge blur based on the chromaticity signal is relatively difficult to detect. Using this characteristic, in the first state where the change of the subject is relatively small, the contour of the subject is difficult to displace and the blurring of the edge is not noticeable. Therefore, averaging processing is performed using the luminance signal to reduce noise. it can. On the other hand, in the second state where the change of the subject is relatively large, the contour of the subject is likely to be displaced. Therefore, by performing the averaging process using the chromaticity signal, noise can be reduced while preventing edge blurring.

  The image input device according to claim 3 is the image input device according to claim 2, wherein the predetermined number of the preceding frames used when the processing means performs an averaging process using a luminance signal is variable. The predetermined number of the preceding frames used when the processing means performs an averaging process using a chromaticity signal is fixed. Human vision has the characteristic that image blur based on luminance signals is relatively easy to detect, but image blur based on chromaticity signals is relatively difficult to detect. Even if the number of frames used is fixed, there is little risk of edge blurring being recognized. On the other hand, the number of frames used for the averaging process using the luminance signal is excluded from the averaging process if there are frames in which the area exceeding the first threshold occupies a large proportion. By making the number of frames to be averaged variable, blurring of edges can be suppressed.

  According to a fourth aspect of the present invention, in the image input device according to the second or third aspect, the chromaticity signal is averaged through a low-pass filter before the averaging process by the processing means or after the averaging process. It is characterized by that. Human vision has the characteristic that image blur based on the luminance signal is relatively easy to detect, but image blur based on the chromaticity signal is relatively difficult to detect, so the chromaticity signal is passed through a low-pass filter. Further noise reduction can be achieved by introducing spatial averaging.

  The image input device according to claim 5 is the image input device according to any one of claims 2 to 4, wherein the luminance signal or the chromaticity signal used when the averaging processing is performed by the processing unit is the processing. The target frame is used as a reference, and weighting is performed according to a time difference from a temporally preceding frame. As the time goes back from the present time to the past, the possibility that the change of the subject in the frame will increase increases. Therefore, by using the processing target frame as a reference, weighting is performed so as to gradually decrease, for example, in accordance with the time difference up to the preceding frame, thereby suppressing the influence of the frame farther than the present time and suppressing blurring of the edge. be able to.

  An image input device according to a sixth aspect is the image input device according to any one of the second to fifth aspects, wherein a difference between the luminance signal of the preceding frame and the luminance signal of the processing target frame is the first. When the second threshold value greater than the first threshold value is exceeded, the detection means classifies the subject change as a third state larger than the second state, and the processing means performs averaging. The processing is not performed. This is because, for example, when a large scene change occurs, averaging cannot be performed using the luminance signal and chromaticity signal of the preceding frame.

  The image input device according to a seventh aspect is the image input device according to any one of the first to sixth aspects, wherein the predetermined region is one pixel. As a result, noise can be accurately reduced for each pixel.

  An image input apparatus according to an eighth aspect is the image input apparatus according to any one of the first to sixth aspects, wherein the predetermined area is a pixel block including a plurality of pixels. Compared with the case where image processing is performed for each pixel, the processing time can be shortened.

An image input device according to claim 9 is provided.
Imaging means for entering subject light and converting it into an image signal;
Separating means for separating the image signal sequentially output from the imaging means into a luminance signal and a chromaticity signal for each frame;
For the luminance signal of the processing target frame in a plurality of temporally continuous frames, the luminance signal corresponding to a predetermined number of preceding frames temporally preceding the processing target frame is weighted with a first weighting amount. The chromaticity signal in the processing target frame is added to the corresponding chromaticity signal in a predetermined number of preceding frames temporally preceding the processing target frame. Processing means for performing addition processing for each predetermined region by performing weighting by a second weighting amount that is equal to or greater than one weighting amount;
Image data forming means for synthesizing the luminance signal and the chromaticity signal added by the processing means and forming image data corresponding to the processing target frame.

  Human vision has a characteristic that image edge blur based on the luminance signal is relatively easy to detect, but image edge blur based on the chromaticity signal is relatively difficult to detect. Therefore, the processing means takes the first luminance signal corresponding to a predetermined number of preceding frames temporally preceding the processing target frame for the luminance signal of the processing target frame in a plurality of temporally continuous frames. In addition to performing addition processing for each predetermined region by weighting according to the weighting amount, the chromaticity signal in the processing target frame corresponds to the predetermined number of preceding frames temporally preceding the processing target frame. If the chromaticity signal is weighted by the second weighting amount that is equal to or greater than the first weighting amount and the addition processing is performed for each predetermined region, noise is effectively suppressed while suppressing blurring of the edge. Can be reduced. The “second weight amount equal to or greater than the first weight amount” means that the first weight amount and the second weight amount in a predetermined number of preceding frames temporally preceding the processing target frame. When there are a plurality of weights, the case where any one of the weighting amounts is equal is included, but the case where all the weighting amounts are equal is excluded.

  The image input device according to claim 10 is the image input device according to claim 9, wherein the processing unit uses the processing target frame as a reference, and determines the first according to a time difference to a temporally preceding frame. The weighting amount of 1 is changed. As the time goes back from the present time to the past, the possibility that the change of the subject in the frame will increase increases. Therefore, based on the processing target frame, according to the time difference up to the preceding frame, for example, by weighting so as to gradually decrease as the time difference increases, the influence of the frame far from the current time is suppressed, Edge blur can be suppressed.

  The image input device according to claim 11 is the image input device according to claim 9 or 10, wherein the processing unit uses the processing target frame as a reference, and according to a time difference up to a temporally preceding frame. The second weighting amount is changed. The effect of the present invention is the same as that of the tenth aspect.

  The image input device according to claim 12 is the image input device according to any one of claims 9 to 11, wherein the processing unit uses the processing target frame as a reference, and a time difference to a temporally preceding frame. Is over the first time, the first weighting amount is set to zero. Thereby, blurring of the edge can be effectively avoided.

  The image input device according to claim 13 is the image input device according to claim 12, wherein the processing unit uses the processing target frame as a reference, and a time difference from a temporally preceding frame to the first frame is the first time. When the second time longer than the time is exceeded, the second weighting amount is set to zero. Thereby, it is possible to effectively reduce noise while effectively avoiding edge blur.

  The image input device according to a fourteenth aspect is the image input device according to any one of the ninth to thirteenth aspects, wherein the predetermined region is one pixel. As a result, noise can be accurately reduced for each pixel.

  An image input apparatus according to a fifteenth aspect is the image input apparatus according to any one of the ninth to thirteenth aspects, wherein the predetermined area is a pixel block including a plurality of pixels. Compared with the case where image processing is performed for each pixel, the processing time can be shortened.

  According to the present invention, it is possible to provide an image input device that can remove noise from moving images while preventing edge blurring with a simple configuration.

It is the schematic of image input device MG1 which concerns on 1st Embodiment. It is a figure which shows the state which compares a luminance value and a threshold value for every pixel. It is a figure which shows the state which compares a luminance value and a threshold value for every pixel block. It is the schematic of image input device MG2 which concerns on 2nd Embodiment. Although it is a graph which shows the 1st weighting amount and 2nd weighting amount which are used for this Embodiment, it goes back in the past as it goes to the horizontal axis right side. Although it is a graph which shows the 1st weighting amount and 2nd weighting amount which are used for the modification of this Embodiment, it goes back in the past as it goes to the horizontal axis right side.

(First embodiment)
FIG. 1 is a schematic diagram of an image input device MG1 according to the first embodiment. Such an image input device MG1 can be used for an in-vehicle camera, but the application is not limited thereto, and may be a surveillance camera, for example. The image input device MG1 includes an imaging device (for example, a CCD) 101 that is an imaging unit, an A / D conversion unit 102, a luminance / chromaticity separation unit 103 that is a separation unit, a luminance difference calculation unit 111 that is a detection unit, and a luminance average unit. 112, a chromaticity averaging unit 113, a luminance / chromaticity synthesis unit 106A, which is an image data forming unit, and a display unit 107. The luminance average unit 112 and the chromaticity average unit 113 constitute processing means. Hereinafter, the coordinates of each pixel in one frame are represented by (x, y).

Subject light forms an image on a plurality of pixels of the image sensor 101 via an RGB color filter (not shown). As a result, the image sensor 101 repeatedly generates an analog color image signal at a predetermined timing, so that the A / D conversion unit 102 converts it into a digital color image signal (color signals dR, dG, dB) for each frame. And output. The luminance / chromaticity separation unit 103 that receives the color image signal receives the color signals dR, dG, and dB input via the image sensor 101 and the A / D conversion unit 102 as shown in the following equation (1). Is converted into a color space including the luminance signal Y and the chromaticity signals Cb and Cr. Here, the chromaticity signal Cb indicates a blue chromaticity signal, and the chromaticity signal Cr indicates a red chromaticity signal.
Y = 0.3dR + 0.59dG + 0.11dB
Cb = −0.17 dR−0.33 dG + 0.5 dB (1)
Cr = 0.5dR-0.42dG-0.08dB
The luminance difference calculation unit 111 calculates luminance signals from a current number of frames F (t−N) retroactive to N frames in the past to the current frame F (t) that is imaged at time t on the time axis. Each pixel (x, y) can be stored. Further, the luminance difference calculation unit 111 calculates the luminance signal of the current frame F (t) and the past frames F (t−1), F (t−2),... F (t−N). When the absolute value of the difference from the luminance signal is obtained for each pixel, the luminance difference is ΔY (t−1) (x, y), ΔY (t−2) (x, y),. ΔY (t−N) (x, y).

  Next, as shown in FIG. 2, the luminance difference calculation unit 111 performs luminance difference ΔY (t−1) (x, y), ΔY (t−2) (x, y), for each pixel of each frame. ... ΔY (t−N) (x, y) is compared with the first threshold th1. Here, when the luminance difference is equal to or smaller than the first threshold th1, the luminance average unit 112 classifies the subject as a first state in which the change in the subject is relatively small, and based on the following equation (2), the luminance Averaging is performed on the signal. Here, the weighting coefficient a1 applied for each frame uses a value used in a second embodiment described later.

  On the other hand, the weighting coefficient a1 is set to zero for frames and pixels whose luminance difference exceeds the first threshold th1. As a result, when the change of the subject becomes relatively large, it is possible to suppress the blurring of the edge by not performing the process of averaging the luminance signal. Needless to say, when the weighting coefficient of a certain frame and pixel is set to zero, the sum of the other weighting coefficients is set to 1.

  Further, the luminance difference calculation unit 111 performs luminance differences ΔY (t−1) (x, y), ΔY (t−2) (x, y),... ΔY (t−) for each frame and pixel. N) (x, y) is compared with a second threshold th2 that is greater than the first threshold th1 (th1 <th2). Here, when the luminance difference exceeds the first threshold th1 and is equal to or less than the second threshold th2, the chromaticity average unit 113 is in the second state because the change in the subject is larger than the first state. Based on the following equation (3), the chromaticity signal is averaged. Here, the weighting coefficient a2 applied for each frame uses a value used in a second embodiment described later. In this example, since the chromaticity signal is averaged even when the luminance difference is equal to or smaller than the first threshold th1, noise can be further reduced.

On the other hand, the frames and pixels whose luminance difference exceeds the second threshold th2 are classified as the third state because the change in the subject is larger than the second state, and the weighting coefficient a2 is set to zero. To do. Thus, when a scene change occurs, it is possible to avoid an image from becoming inappropriate by performing an erroneous averaging process on frames of different scenes. Thereafter, the luminance / chromaticity combining unit 106A receives the signal SY (t) averaged by the luminance averaging unit 112 and the signals SCr (t) and SCb (t) averaged by the chromaticity averaging unit 113. Based on the above, the color signals dR, dG, and dB are synthesized according to the following equation (1 ′) and input to the display unit 107, so that they can be displayed as a frame at the time point t. A moving image can be reproduced by switching a plurality of frames processed in the same manner.
dR = SY (t) + 1.402 · SCr (t)
dG = SY (t) −0.714 · SCr (t) −0.344 · SCb (t) (1 ′)
dB = SY (t) + 1.772 · SCb (t)
FIG. 3 is a diagram for explaining a modification of the present embodiment. In the above-described embodiment, averaging processing is performed for each pixel. In this modification, for example, 16 × 16 pixels in one frame are defined as one pixel block (represented by coordinates (X, Y)), and the frame And luminance differences ΔY (t−1) (X, Y), ΔY (t−2) (X, Y),... ΔY (t−N) (X, Y) averaged in pixel block units ) Is compared with the first threshold th1 and the second threshold th2. According to this modification, the number of comparisons is reduced, so that the processing speed can be significantly increased.

  In the embodiment described above, the chromaticity signal can be averaged by determining the number of frames using a fixed number of frames and the luminance signal using a threshold value (that is, variable). For example, with respect to the luminance signal of a frame to be processed, N = 10 when the luminance signal difference first exceeds the threshold retroactively is 11th, and N = 14 when it is the 15th. The above processing is performed. Since fluctuations in luminance have a great influence on edge blurring, averaging of luminance can be performed with high accuracy. Since variation in chromaticity has little effect on edge blurring, the amount of calculation can be reduced with a fixed number of frames.

  In the embodiment described above, the chromaticity signal can be combined with a spatial average by passing it through a low-pass filter before or after the averaging processing by the processing means. Since the variation in chromaticity has little influence on the edge blur, the noise can be further reduced by introducing a spatial average.

  In addition, for each pixel or pixel block, the number of luminance signal average frames determined by the threshold is compared with the average number of frames of chromaticity signals that are fixed values (the maximum is averaged up to this number of frames). It is also possible to set the number of frames to average the number. When the number of frames determined by the threshold and the number of frames fixed are mixed, the smaller one can be selected to reduce the influence of edge blurring.

Further, when the luminance difference is equal to or smaller than the first threshold th1, the chromaticity signal averaging process may be not performed. As a result, the processing speed can be increased.
(Second Embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 4 is a schematic diagram of an image input device MG2 according to the second embodiment. Such an image input device MG2 can be used for an in-vehicle camera, but the application is not limited thereto, and may be a surveillance camera, for example. The image input device MG2 includes an imaging device (for example, a CCD) 101 that is an imaging unit, an A / D conversion unit 102, a luminance / chromaticity separation unit 103 that is a separation unit, a luminance weighted addition unit 104, a chromaticity weighted addition unit 105, A luminance / chromaticity synthesis unit 106B and a display unit 107, which are image data forming means, are provided. The luminance weighted addition unit 104 and the chromaticity weighted addition unit 105 constitute processing means. Hereinafter, the coordinates of each pixel in one frame are represented by (x, y).

Subject light forms an image on a plurality of pixels of the image sensor 101 via an RGB color filter (not shown). As a result, the image sensor 101 repeatedly generates an analog color image signal at a predetermined timing, so that the A / D conversion unit 102 converts it into a digital color image signal (color signals dR, dG, dB) for each frame. And output. The luminance / chromaticity separation unit 103 to which the color image signal is input, as shown in Expression (1), converts the color signals dR, dG, and dB into a color space including the luminance signal Y and the chromaticity signals Cb and Cr. Convert. Here, the chromaticity signal Cb indicates a blue chromaticity signal, and the chromaticity signal Cr indicates a red chromaticity signal.
Y = 0.3dR + 0.59dG + 0.11dB
Cb = −0.17 dR−0.33 dG + 0.5 dB (1)
Cr = 0.5dR-0.42dG-0.08dB
The luminance weighted addition unit 104 performs a predetermined number of frames F (t) retroactive to the current frame F (t) that is imaged at time t on the time axis (for example, N / 30 seconds as 30 fps). -N) can be stored for each pixel (x, y), and has weighting coefficients a1 (t) to a1 (t-N) corresponding to the frames as shown in FIG. Yes. On the other hand, the chromaticity weighted addition unit 105 outputs chromaticity signals up to a predetermined number of frames F (t−N) retroactive to the current frame F (t) to pixels (x, y). Each of them has a weighting coefficient a2 (t) to a2 (t−N) as shown in FIG. The weighting coefficients a1 and a2 gradually decrease from the current frame F (t) to the past frames F (t-1), F (t-2),... {Ie, a1 (t− N) <a1 (t-N + 1) <... <a1 (t-1) <a1 (t) and a2 (t-N) <a2 (t-N + 1) <... <a2 (t-1) <a2 (t )}, Zero in a predetermined number of frames. However, a1 (t−N) + a1 (t−N + 1) +... + A1 (t−1) + a1 (t) = 1 and a2 (t−N) + a2 (t−N + 1) +. 1) + a2 (t) = 1. The weighting coefficients a1 and a2 are always a1 <a2 except for the current frame F (t) and a predetermined number of frames F (t−N) retroactively.

  In the current frame F (t), a1> a2.

  The luminance weighted addition unit 104 receives the luminance signal Y (t) (x, y) for the current frame F (t), and for each pixel (x, y) based on the following equation (4), The luminance signals from a predetermined number of frames F (t−N) retroactive to N frames F (t) are added to obtain SY (t) (x, y).

  On the other hand, the chromaticity weighted addition unit 105 inputs chromaticity signals Cr (t) (x, y) and Cb (t) (x, y) for the current frame F (t), and the following (5) Based on the equation, for each pixel (x, y), the chromaticity signals from the current frame F (t) to a predetermined number of frames F (t−N) retroactive to the past N frames are added, and SCr ( t) (x, y), SCb (t) (x, y) are obtained.

The luminance / chromaticity synthesis unit 106B obtains (1 ′) the SY (t) output from the luminance weighted addition unit 104 and the SCr (t) and SCb (t) output from the chromaticity weighted addition unit 105. The color signals dR, dG, and dB are obtained according to the equation, and these are combined and input to the display unit 107, so that they can be displayed as a frame at the time axis t. A moving image can be reproduced by switching a plurality of frames processed in the same manner.
dR = SY (t) + 1.402 · SCr (t)
dG = SY (t) −0.714 · SCr (t) −0.344 · SCb (t) (1 ′)
dB = SY (t) + 1.772 · SCb (t)
According to the present embodiment, the luminance weighted addition unit 104 has a predetermined number of preceding the processing target frame for the luminance signal Y (t) (x, y) in the processing target frame F (t) (x, y). Corresponding luminance signals Y (t−1) (x, y) to Y (t−N) (x, y) in frames F (t−1) (x, y) to F (t−N) (x, y). y) is weighted according to the first weighting amount a1 (t-1) (x, y) to a1 (t-N) (x, y), and addition processing is performed (processing of equation (4)). The chromaticity weighted addition unit 105 adds the chromaticity signals Cr (t) (x, y) and Cb (t) (x, y) in the processing target frame F (t) (x, y) to the processing target frame. Corresponding chromaticity signal Cr (t−1) in a predetermined number of preceding frames F (t−1) (x, y) to F (t−N) (x, y) x, y) to Cr (t−N) (x, y), Cb (t−1) (x, y) to Cb (t−N) (x, y), respectively. t−1) (x, y) to a1 (t−N) (x, y) or more of the second weighting amounts a2 (t−1) (x, y) to a2 (t−N) (x, Since the addition processing is performed by weighting according to y) (the processing of equation (5)), the second weighting amount is suppressed while suppressing blurring of the edge by relatively reducing the first weighting amount. The noise can be reduced by relatively increasing. Although the above weighting is performed in units of pixels, it may be performed in units of 16 × 16 pixels, for example.

  FIG. 6 is a graph showing the first weighting amount and the second weighting amount used in the modification of the present embodiment, which goes back to the past as it goes to the right side of the horizontal axis. In the 30 fps specification, one frame is switched in 1/30 seconds. That is, when N1 seconds go back from the current t seconds in the past, the number of frames during that time is N1 · 30 frames. Assuming that the current frame is F (t), the frame at the time point that goes back in the past for N1 seconds is taken as F (t-n1), and the frame at the time point that goes back in the past N2 seconds is taken as F (t-n2). In the present modification, the first weighting amount a1 decreases linearly from the current frame F (t), which is imaging at the time point t, on the time axis, and then reaches a predetermined time that is N1 seconds past the first time. It is zero before the fewth frame F (t−n1). On the other hand, the second weighting amount a2 is constant, but a predetermined number of frames F () (N2 seconds past the second time) from the current frame F (t) that is imaged at time t on the time axis. t−n2) Before zero. It is preferable that n1 <n2. Human vision suppresses edge blur by using the characteristic that image edge blur based on luminance signal is relatively easy to detect, but image edge blur based on chromaticity signal is relatively hard to detect. However, noise is reduced.

  The present invention can be applied to in-vehicle cameras, surveillance cameras, and the like, but the application is not limited thereto.

DESCRIPTION OF SYMBOLS 101 Image pick-up element 102 A / D conversion part 103 Luminance / chromaticity separation part 104 Luminance weighted addition part 105 Chromaticity weighted addition part 106A, 106B Luminance / chromaticity synthesis part 107 Display part 111 Luminance difference calculation part 112 Luminance average part 113 Color Degree averaging unit MG1, MG2 Image input device

Claims (15)

Imaging means for entering subject light and converting it into an image signal;
Separating means for separating the image signal sequentially output from the imaging means into a luminance signal and a chromaticity signal for each frame;
Changes in the subject between a processing target frame in a plurality of temporally continuous frames and a frame temporally preceding the processing target frame are detected for each predetermined region, and classified into a plurality of states according to the subject change Detecting means for
For at least one of the luminance signal and chromaticity signal in the processing target frame separated by the separation means, using at least one of the luminance signal and chromaticity signal corresponding to a predetermined number of the preceding frames, Processing means for performing different averaging processes according to the classification by the detection means for each predetermined region;
An image input device comprising: image data forming means for combining the luminance signal and chromaticity signal processed by the processing means to form image data corresponding to the processing target frame. When the difference between the luminance signal of the preceding frame and the luminance signal of the processing target frame is equal to or less than a first threshold, the detection unit classifies the change of the subject as being in the first state. The processing means performs an averaging process using the luminance signal,
When the difference between the luminance signal of the preceding frame and the luminance signal of the processing target frame exceeds the first threshold value, the detection unit has a change in the subject larger than that in the first state. The image input apparatus according to claim 1, wherein the image input device is classified as being in a second state, and the processing unit performs an averaging process using the chromaticity signal.   The predetermined number of the preceding frames used when the processing unit performs the averaging process using the luminance signal is variable, and is used when the processing unit performs the averaging process using the chromaticity signal. The image input apparatus according to claim 2, wherein the predetermined number of the preceding frames is fixed.   The image input apparatus according to claim 2 or 3, wherein the chromaticity signal is averaged through a low-pass filter before or after the averaging process by the processing means.   The luminance signal or chromaticity signal used when the averaging process is performed by the processing unit is weighted according to a time difference to a frame temporally preceding the processing target frame. The image input device according to claim 2.   When the difference between the luminance signal of the preceding frame and the luminance signal of the processing target frame exceeds a second threshold value that is larger than the first threshold value, the detection means detects that the subject has changed. The image input device according to claim 2, wherein the image input device is classified as a third state larger than the state 2 and the processing means does not perform an averaging process.   The image input apparatus according to claim 1, wherein the predetermined area is one pixel.   The image input apparatus according to claim 1, wherein the predetermined area is a pixel block including a plurality of pixels. Imaging means for entering subject light and converting it into an image signal;
Separating means for separating the image signal sequentially output from the imaging means into a luminance signal and a chromaticity signal for each frame;
For the luminance signal of the processing target frame in a plurality of temporally continuous frames, the luminance signal corresponding to a predetermined number of preceding frames temporally preceding the processing target frame is weighted with a first weighting amount. The chromaticity signal in the processing target frame is added to the corresponding chromaticity signal in a predetermined number of preceding frames temporally preceding the processing target frame. Processing means for performing addition processing for each predetermined region by performing weighting by a second weighting amount that is equal to or greater than one weighting amount;
An image input device comprising: image data forming means for combining the luminance signal and the chromaticity signal added by the processing means to form image data corresponding to the processing target frame.   The image input apparatus according to claim 9, wherein the processing unit changes the first weighting amount according to a time difference from a temporally preceding frame to the processing target frame as a reference.   11. The image input according to claim 9, wherein the processing unit changes the second weighting amount according to a time difference from a temporally preceding frame to the processing target frame as a reference. apparatus.   The processing means sets the first weighting amount to zero when a time difference to a frame preceding in time exceeds a first time with the processing target frame as a reference. Item 12. The image input device according to any one of Items 9 to 11.   The processing means sets the second weighting amount to zero when the time difference up to the frame preceding in time exceeds the second time longer than the first time with the processing target frame as a reference. The image input device according to claim 12, wherein:   The image input device according to claim 9, wherein the predetermined area is one pixel.   The image input apparatus according to claim 9, wherein the predetermined area is a pixel block including a plurality of pixels.