JP2003259211A - Image processor and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prepare an output image in which an object can be clearly distinguished from its background by appropriately improving the contrast of a frame image picked up at a high frame rate. <P>SOLUTION: An image analysis processing part 5 applies a frequency analysis to the frame image picked up at a high frame rate by a CCD (charge coupled device) camera 2 to specify an area of low contrast in the frame image. An output image preparation processing part 6 adds image data of the area of low contrast in the frame image for a plurality of frames to output an output image with improved contrast. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method for processing a frame image captured at a high frame rate to create an output image.

[0002]

2. Description of the Related Art Conventionally, CCD (Charge Coupled Devic)
e) As an apparatus for synthesizing a plurality of images picked up by an image pickup means such as a camera to create one output image, for example, an image synthesizing apparatus as disclosed in Japanese Patent Laid-Open No. 8-154201 has been proposed. There is.

In the image synthesizing apparatus disclosed in Japanese Unexamined Patent Publication No. 8-154201, data of an image (with long charge accumulation time) taken by opening a diaphragm is used for a dark portion of a subject, and for a bright portion thereof. By using the data of the image captured with a narrow aperture (short charge accumulation time), the output image is composed by the weighted sum of the image captured with the aperture closed and the image captured with the aperture open in the middle of both. , An image of a subject with a high dynamic range can be output without saturation or blackout.

Further, in this image synthesizing apparatus, the gains of the images picked up by different apertures are accurately adjusted according to the level of the image signal, so that the discontinuity in the overlapping portion of the image signal after weighted addition is achieved. , So that the image signal level is continuous in all ranges,
The quality of the output image is improved.

[0005]

However, in the conventional image synthesizing apparatus as described above, the contrast of the output image is determined by the contrast of the images used for synthesizing, and a higher contrast cannot be obtained. There is.

In other words, this image synthesizing apparatus alleviates the difference in signal level between the bright portion and the dark portion in the entire image by synthesizing images having different apertures, thereby expanding the dynamic range. The main purpose is to select an image to be used for composition based on the brightness of the subject. For this reason, for example, the contrast in the bright part is limited to the contrast of the image with the aperture stopped, and the contrast in the dark part is limited to the contrast of the image with the aperture stopped, and the contrast of these bright and dark parts is limited. There is a problem that it is difficult to improve the internal contrast and to clearly separate the subject and the background in each of these regions.

Further, in general, an image with a wide aperture can obtain a higher contrast than an image with a narrow aperture. However, in the conventional image synthesizing apparatus as described above, the saturation of pixel values is prevented. In order to prevent this from occurring, an image with a narrow aperture is used for the bright portion of the entire image, so it is difficult to obtain a high contrast especially for the bright portion.

The present invention was devised in view of the above-mentioned conventional circumstances, and improves the contrast regardless of the brightness of the image so that an output image can be clearly distinguished between the subject and the background. An object is to provide an image processing device and an image processing method to be created.

[0009]

According to a first aspect of the present invention, image pickup means for picking up a frame image in a cycle shorter than the output cycle of an output image, and frequency analysis of the frame image picked up by the image pickup means. , Frequency analysis means for specifying a low contrast area in the frame image based on the analysis result, and image data of a plurality of frames of the area specified as a low contrast area by the frequency analysis means are added for a plurality of frames, An image processing apparatus comprising: an output image creating means for creating the output image.

The invention described in claim 2 is the same as claim 1.
In the image processing device according to the item (3), the frequency analysis unit specifies a region in which a spectral intensity exceeding a predetermined threshold value is not obtained in a predetermined frequency band as a low contrast region.

The invention described in claim 3 is the same as claim 1.
Alternatively, in the image processing device according to the second aspect, the output image creating unit adds a plurality of frames of image data in the low contrast area, and then converts a DC component of the image data into a DC component of image data in an adjacent area. It is characterized by performing a correction process to bring them closer.

The invention according to claim 4 is the same as claim 3
In the image processing device described in the paragraph 1, the output image creating means calculates a correction amount of the correction processing based on a spectrum intensity indicating a DC component of the image data in the low contrast region. .

According to a fifth aspect of the present invention, an image pickup means capable of picking up a frame image in a cycle shorter than the output cycle of the output image, and a frequency analysis of the frame image picked up by the image pickup means, and the analysis is performed. Frequency analysis means for specifying a low contrast area in the frame image based on the result, and reset of accumulated charge in the imaging area of the imaging means corresponding to the area specified as the low contrast area by the frequency analysis means The image forming apparatus comprises: an accumulation control unit that delays the timing by a plurality of frames; and an output image creating unit that creates the output image based on the image output from the imaging unit according to the control of the accumulation controlling unit. Image processing apparatus.

Further, the invention according to claim 6 acquires the frame image from an image pickup means for picking up the frame image at a cycle shorter than the output cycle of the output image, frequency-analyzes the acquired frame image, and A region of low contrast in the frame image is specified based on an analysis result, and image data of a region of low contrast in the frame image is added for a plurality of frames to create the output image. This is an image processing method.

Further, in the invention described in claim 7, the frame image is acquired from the image pickup means capable of capturing the frame image in a cycle shorter than the output cycle of the output image, and the acquired frame image is subjected to frequency analysis, An area having a low contrast in the frame image is specified based on the analysis result, and the reset timing of the accumulated charges in the imaging area of the imaging unit corresponding to the area having a low contrast in the frame image is delayed by a plurality of frames. The image processing method is characterized in that control is performed and the output image is created based on the image output from the imaging unit according to the control.

[0016]

According to the image processing apparatus of the present invention, an area having a low contrast in a frame image is specified by frequency analysis, and image data of the area having a low contrast is added for a plurality of frames. Since the pixel value difference between the subject and the background in the area is increased to improve the contrast, the contrast of the low contrast portion is appropriately improved without being influenced by the brightness of the image, It is possible to obtain an output image in which the subject and the background can be clearly distinguished.

In particular, in this image processing apparatus, when a region in which a spectral intensity exceeding a predetermined threshold value is not obtained in a predetermined frequency band is specified as a region with low contrast, the frequency band of interest is appropriately selected. By setting, it is possible to accurately specify a region with low contrast.

Further, in this image processing apparatus, after the image data of the low contrast region is added for a plurality of frames, the DC component of this image data is adjacent to the adjacent region based on, for example, the spectrum intensity indicating the DC component of this image data. If the correction processing is performed so as to approach the DC component of the image data of, the inconvenience of a large difference in pixel value between the area for improving the contrast and other areas adjacent to this area can be suppressed. As a result, the deterioration of the image quality of the output image can be effectively prevented.

Further, according to another image processing apparatus of the present invention, an area having a low contrast in the frame image is specified by frequency analysis, and the accumulated charge in the imaging area of the image pickup means corresponding to the area having a low contrast is identified. By delaying the reset timing by a plurality of frames, the pixel value difference between the subject and the background in this area is increased and the contrast is improved.
It is possible to obtain an output image in which the subject and the background can be clearly distinguished by appropriately improving the contrast in a low contrast portion without being influenced by the brightness of the image.

Further, according to the image processing method of the present invention, a region of low contrast in the frame image is specified by frequency analysis, and the image data of the region of low contrast is added for a plurality of frames. Since the difference in pixel value between the subject and the background in the image is increased to improve the contrast, the contrast of the low contrast part is appropriately improved without depending on the brightness of the image, and the contrast between the subject and the background is improved. It is possible to obtain an output image that can be clearly distinguished from and.

According to another image processing method of the present invention, a region of low contrast in the frame image is specified by frequency analysis, and the accumulated charge in the image pickup region of the image pickup means corresponding to the region of low contrast is identified. By delaying the reset timing by a plurality of frames, the pixel value difference between the subject and the background in this area is increased to improve the contrast, so that the contrast is low regardless of the brightness of the image. By appropriately improving the contrast of the part, it is possible to obtain an output image in which the subject and the background can be clearly distinguished.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First Embodiment) FIG. 1 is a block diagram showing a structural example of an image processing apparatus to which the present invention is applied. The image processing apparatus 1 shown in FIG. 1 is used, for example, as an apparatus mounted on a vehicle to create an output image for monitoring the surroundings of the vehicle, and includes a CCD camera 2 and an A / D.
Converter 3, frame memory 4, image analysis processing unit 5
And an output image creation processing unit 6 and a D / A converter 7. When monitoring the surroundings of a vehicle based on an image, it is particularly important to be able to properly judge the type of the object shown as the image. Therefore, in the image processing apparatus 1 to which the present invention is applied, an output image having a high contrast is created so that an object and a background can be clearly distinguished in the output image. The image processing device 1 to which the present invention is applied is not limited to the device for monitoring the surroundings of a vehicle, but can be widely applied as a device for satisfying the demand when a high-contrast image is required. .

The CCD camera 2 picks up an image of a subject as a frame image. Particularly, in the image processing apparatus 1 to which the present invention is applied, the CCD camera 2 is
It is configured so that an image of a subject can be taken at a high frame rate. That is, the CCD camera 2 is adapted to capture a frame image at a cycle shorter than the output cycle of the output image. It is designed to capture ten frame images. The frame image captured by the CCD camera 2 is supplied to the A / D converter 3 as a voltage value signal for each pixel forming each frame image.
As the means for picking up the image of the subject, an image pickup device other than the CCD camera 2 may be used as long as it can pick up a frame image in a cycle shorter than the output cycle of the output image.

The A / D converter 3 converts the voltage value signal supplied from the CCD camera 2 into an analog / digital signal to obtain digital image data representing each frame image. The digital image data converted from the voltage value signal by the A / D converter 3 is supplied to the frame memory 4. The A / D converter 3 may be integrated with the CCD camera 2.

The frame memory 4 stores and holds the digital image data output from the A / D converter 3 in units of frames.

The image analysis processing section 5 performs signal processing such as Fourier transform on the digital image data held in the frame memory 4 for each frame, that is, the image data of the frame image picked up by the CCD camera 2. ,
The frequency of the frame image is analyzed.

More specifically, the image analysis processing section 5 has a C
The frame image captured by the CD camera 2 is divided into a plurality of small areas, and each divided area is subjected to, for example, a two-dimensional Fourier transform process to calculate the spatial frequency in each divided area. Then, paying attention to a predetermined frequency band, it is determined for each divided region whether or not a spectrum intensity exceeding a predetermined threshold is obtained in this frequency band, and a spectrum intensity exceeding a predetermined threshold is obtained in the frequency band of interest. The area that has not been obtained is specified as a low contrast area (hereinafter referred to as a low contrast area) in the frame image captured by the CCD camera 2.

The output image creation processing unit 6 adds the image data of the low contrast region specified by the image analysis processing unit 5 for a plurality of frames to create the image data of the output image. Specifically, the output image creation processing unit 6 acquires, for example, the image data of the first frame image of a plurality of continuous frame images corresponding to one output image from the frame memory 4 and stores it in the internal memory. The image data of the low-contrast area specified by the image analysis processing unit 5 is extracted from the image data of the subsequent frame images which are held and are sequentially supplied to the frame memory 4, and the extracted image data of the low-contrast area is set at the top. The image data of the frame image is sequentially added, and finally the image data of the output image in which the contrast in the low contrast region is improved is created. Here, the number of frames to be added to the low contrast area, that is, the number of frames of image data to be added to the low contrast area is determined according to the degree of low contrast in the low contrast area.

The D / A converter 7 includes an output image creation processing unit 6
The digital image data of the output image created by is converted from digital to analog and output as an image signal. The image signal output from the D / A converter 7 is
For example, it is supplied to a display device such as a liquid crystal display. Then, the display device is driven based on this image signal, so that an image with improved contrast is displayed on the display device.

Here, the operation of the image processing apparatus 1 to which the present invention configured as described above is applied will be described in detail with reference to specific examples.

When the image processing apparatus 1 is activated, an image of an object to be imaged (reflected light from the object) is formed on the image pickup section of the CCD camera 2. Then, the reflected light from the subject is photoelectrically converted from photons into charges in the image pickup section of the CCD camera 2, and the charges accumulated during the charge accumulation time of the CCD camera 2 are converted into a voltage to form an image. It is output as a voltage value signal indicating a voltage value (pixel value) for each pixel. Here, since the intensity of light incident on the image pickup unit of the CCD camera 2 and the amount of charge accumulated in this image pickup unit are in a proportional relationship, the voltage value signal output from the CCD camera 2 is the brightness of the subject. Will be reflected. That is, the bright part of the subject is shown as a high voltage value, and the dark part of the subject is shown as a low voltage value.

When the charge storage time is set to be long, the charge storage capacitor is saturated in the image pickup area of the CCD camera 2 where the reflected light from the bright part of the subject is incident. In some cases, the voltage value signal that appropriately reflects the brightness of the subject cannot be obtained. However, in the image processing apparatus 1 to which the present invention is applied, as described above, the CCD camera 2 has a structure for capturing an image of a subject at a high frame rate, and this frame rate limits the upper limit of the charge storage time. As a result, the charge storage time is set to be short, so that the above-mentioned problem of capacitor saturation can be avoided.

The voltage value signal output from the CCD camera 2 is converted into digital image data representing a frame image by the A / D converter 3 and stored in the frame memory 4. Here, in the image processing apparatus 1 to which the present invention is applied, as described above, the CCD camera 2 captures a frame image at a cycle shorter than the output cycle of the output image, so that the frame memory 4 stores the frame image. , Output image creation processing unit 6
Image data indicating a plurality of frame images are continuously supplied and accumulated while one output image is output from the. Hereinafter, a set of a plurality of frame images stored in the frame memory 4 for each output cycle of such output images will be referred to as a frame image group for convenience. Each frame image forming this frame image group is an image continuously captured by the CCD camera 2 at a high frame rate, and the change amount of the image between each frame image is extremely small.

The image data of the first frame image of the frame image group is taken into the image analysis processing section 5. The image analysis processing unit 5 frequency-analyzes the image data of the frame image captured from the frame memory 4 for each small area, and specifies the low contrast area in this frame image based on the analysis result.

Explaining with a concrete example, for example, C
When the frame image as shown in FIG. 2 is captured by the CD camera 2, the image analysis processing unit 5 first divides the frame image into a plurality of small areas. Then, for example, a two-dimensional Fourier transform process is performed on each region to create a Fourier transform image showing the spatial frequency for each region. It should be noted that how many small areas the frame image is divided into may be appropriately set according to the processing capability of the image analysis processing unit 5.

The spatial frequency represented by the Fourier transform image is
It is proportional to the fineness of the pattern in the original image before conversion.
That is, an image containing many fine patterns has high spatial frequency components, while an image not containing fine patterns has only low spatial frequency components, and the spatial frequency of a uniform image without patterns has only DC components. Becomes Specifically, for example, when the Fourier transform processing is performed on the original image shown in FIG. 3A, a Fourier transformed image as shown in FIG. 3B is obtained, and the original image shown in FIG. When the Fourier transform processing is performed on, a Fourier transform image as shown in FIG. 4B is obtained. Here, FIG. 3B and FIG.
In the Fourier transform image shown in (b), the center is a DC component of frequency, and the frequency becomes higher as the distance from the center increases. Further, the spectral intensity of the spatial frequency indicates the number of frequency components included in the image, and in the Fourier transform images shown in FIGS. 3B and 4B, the portion with high spectral intensity is represented by a dark point. It represents.

Since an unclear image as shown in FIG. 3 (a) has many low frequency components, the Fourier transform image of FIG. 3 (b) obtained from this image has many frequency components near the center and the Fourier transform image near the center. The spectral intensity is high. On the other hand, when the image becomes clear to some extent as shown in FIG. 4A, it has a higher frequency component than the image of FIG. In the Fourier transform image of (b), high spectral intensity is obtained even in high frequency components distant from the center.

As described above, the fineness of the pattern in the image can be judged from the intensity distribution of the Fourier transform image. An image with low contrast hardly contains fine patterns and almost no high frequency components are observed. Therefore, by determining whether or not high spectral intensity can be obtained in a portion away from the center of the Fourier transform image, this image The contrast of can be determined.

Therefore, the image processing apparatus 1 to which the present invention is applied
The image analysis processing unit 5 divides the frame image captured by the CCD camera 2 into small regions, creates a Fourier transform image for each region, and based on this Fourier transform image, in a predetermined frequency band. The low contrast region in the frame image is specified by determining whether or not the spectral intensity exceeding a predetermined threshold value can be obtained.

Explaining in the above-mentioned example, for example, when a Fourier transform image as shown in FIG. 5 is obtained as a result of Fourier transform of the area A in FIG. 2, the image analysis processing unit 5 uses the Fourier transform image in this Fourier transform image. An area (a circular area in FIG. 5) that is apart from the center to some extent is determined as the frequency band to be determined. Then, the spectrum intensity for each frequency is obtained from this Fourier transform image, and it is determined whether or not the spectrum intensity exceeding the preset threshold is obtained in the frequency band of the determination target, and the threshold preset in the frequency band of the determination target. When the spectrum intensity exceeding 10 is not obtained, this region is specified as the low contrast region.

For simplicity, one-dimensional description will be given. When the spectrum intensity for each frequency of the Fourier transform image shown in FIG. 5 is analyzed, the intensity distribution as shown in FIG. 6 is obtained. Focusing on the frequency bands f0 to f0 ′ determined as the determination target in the intensity distribution of FIG. 6, the spectrum intensity exceeding the preset threshold p0 is not obtained in this frequency band f0 to f0 ′. . Therefore, in such a case, the image analysis processing unit 5 determines that the area A in FIG.
Is specified as a low contrast region. The frequency bands f0 to f0 ′ to be judged and the values of the threshold value p0 may be appropriately set according to the contrast required for the output image.

The image analysis processing unit 5 frequency-analyzes the first frame image of the frame image group,
When the low-contrast region in this frame image is specified, the output image creation processing unit 6 adds the image data of this low-contrast region for a plurality of frames and enlarges the pixel value difference in this low-contrast region to increase the contrast. Take action to improve.

Specifically, for example, as shown in FIG.
A frame image group corresponding to one output image is composed of the frame images F1, F2, F3, F4 and F5, and the first frame image F1 of this frame image group is frequency analyzed by the image analysis processing unit 5, When the area a, the area b, and the area c in the frame image F1 are respectively specified as the low contrast areas, the output image creation processing unit 6 first starts from the second frame image F2 to the area a, the area b, and the area b. The image data of c is extracted, and the extracted image data of the areas a, b, and c is added to the image data of the leading frame image F1. As a result, the difference in pixel value in the regions a, b, and c specified as the low contrast region is doubled, and the contrast of the regions a, b, and c is improved.

When the image data of the area a, the area b and the area c in the second frame image F2 is added to the first frame image F1, the image analysis processing section 5 then causes the area a and the area b to be added. The frequency analysis is performed again on the frame image to which the image data of the region c is added (hereinafter referred to as the added frame image), and the low contrast region in the added frame image is specified. Here, for example, for the region b, the focused frequency band f0 to f0 ′
When it is determined that the spectral intensity exceeding the threshold value p0 is obtained in, the area b is excluded from the low contrast area. Then, for example, when it is determined that the spectrum intensity exceeding the threshold value p0 is not yet obtained in the focused frequency bands f0 to f0 ′ for the regions a and c, the regions a and c are added to the post-addition frame. Identified as a low contrast area in the image.

When the area a and the area c are specified as the low contrast area, the output image creation processing unit 6 then proceeds to 3
The image data of the area a and the image data of the area c are extracted from the first frame image F3, and the image data of the extracted area a and the image data of the area c are added to the image data of the frame image after addition. As a result, the difference between the pixel values in the regions a and c identified as the low contrast region is further doubled, and the contrast between the regions a and c is further improved.

When the image data of the areas a and c in the third frame image F3 is added to the post-addition frame image, the image analysis processing unit 5 then outputs the image data of the areas a and c. The frequency analysis is performed again on the added frame image after addition, and the low contrast region in the frame image after addition is specified. Here, for example, for the region a, the frequency bands f0 to f of interest
When it is determined that the spectral intensity exceeding the threshold value p0 is obtained at 0 ', this region a is excluded from the low contrast region. Then, for example, when it is determined that the spectrum intensity exceeding the threshold value p0 is not yet obtained in the frequency band f0 to f0 ′ of interest for the region c, this region c has a low contrast in the post-addition frame image. Identified as a region.

When the area c is specified as the low contrast area, the output image creation processing unit 6 next extracts the image data of the area c from the fourth frame image F4, and the extracted image data of the area c. Is added to the image data of the frame image after addition. The above processing is repeated until the final frame image of the frame image group (here, the fifth frame image F5) is obtained, and the finally obtained frame image is set as the output image. As a result, the contrast of each area specified as the low contrast area in the first frame image F1 of the frame image group is improved, and an output image that can clearly distinguish the object appearing in these areas from the background is created. become.

However, the final frame image obtained within the output cycle of the output image, that is, the image data of the final frame image of the frame image group is added, and the area which is still the low contrast area is further increased. Is not added. Such an extremely low contrast area is almost an area showing only a uniform background without any change. Therefore, it is considered that there is no problem even if the contrast of such an area in the output image is low. To be

Here, the flow of processing in the image analysis processing unit 5 and the output image creation processing unit 6 for realizing the above-described improvement of image contrast will be described with reference to the flowchart of FIG.

First, in step S1, the image analysis processing unit 5 sets the n-th divided area S in the frame image.
Fourier transform is performed on (n) to create a Fourier transform image F (n). Next, in step S2, the spectrum intensity for each frequency is obtained from the Fourier transform image F (n), and it is determined whether the spectrum intensity exceeding the predetermined threshold value p0 is obtained in the predetermined frequency band f0 to f0 ′. To be done.

Here, when the spectral intensity exceeding the predetermined threshold value p0 is obtained in the predetermined frequency band f0 to f0 ', it is determined that the n-th divided area S (n) has a high contrast. It is determined, and in step S3, the load W (n) when performing the addition process is set to zero. On the other hand, when the spectrum intensity exceeding the predetermined threshold value p0 is not obtained in the predetermined frequency band f0 to f0 ′, the nth divided area S (n) is determined to be a low contrast area, and in step S4. , The load W (n) at the time of performing the addition process is set to 1.

When the load W (n) for performing the addition process is set, next, in step S5, the load W for performing the addition process on all the divided areas in the frame image.
It is determined whether or not (n) has been set, and if there is a divided area for which load setting has not been performed, then n = n + 1 is set in step S6 and the processing from step S1 is repeated.

On the other hand, when the weight W (n) for performing the addition processing is set for all the divided areas in the frame image, next, in step S7, the output image creation processing unit 6 causes the frame image to be processed. The image data of the low-contrast region is added to the image data of 1 to create a frame image after the addition. Here, when the image data of the original frame image is I (t) and the image data of the divided area S (n) is S (n, t), the image data I (t + 1) of the frame image after addition is as follows. It is calculated by Equation 1.

I (t + 1) = I (t) + W (n) × S (n, t) Equation 1 The image data of the low contrast area is added to the image data of the frame image to create a frame image after addition. Then, in step S8, it is determined whether the image output timing has been reached. Then, if the image output timing has not been reached, that is, if it is within the image output cycle, the process returns to step S1 and the same processing is repeatedly performed on the post-addition frame image.

On the other hand, if the image output timing has been reached, then in step S9, the post-addition frame image created in step S7 is output from the output image creation processing section 6 as an output image, and a series of Processing ends.

In the above example, the frequency analysis is performed on all the divided areas in the frame image, and the weight for performing the addition process on all the divided areas is set. For example, it is possible to specify in advance the divided areas that are considered to be important in the frame image and omit the processing for the other divided areas. In this case, the processing load on the image analysis processing unit 5 and the output image creation processing unit 6 can be reduced.

In the image processing apparatus 1 to which the present invention is applied, the image analysis processing unit 5 and the output image creation processing unit 6 perform the above-described series of processing, so that the contrast in the low contrast region in the frame image is improved. Since it is improved, it is possible to obtain an output image in which the subject and the background can be clearly distinguished, as shown in FIG. 9, for example. The output image shown in FIG. 9 is obtained as a result of adding the image data to the low-contrast region A in the frame image shown in FIG. As is clear from comparison with the low contrast area A in the frame image shown in 2, it is understood that the image processing apparatus 1 to which the present invention is applied can obtain an output image with improved contrast in the low contrast area.

Further, in the image processing apparatus 1 to which the present invention is applied, it is determined by the frequency analysis of the image whether or not the contrast is obtained. Therefore, the contrast is low regardless of the brightness of the image. A region can be well identified and the contrast of this region can be improved appropriately, for example in a dark part of a scene where a wide dynamic range is needed, from bright places to dark places inside tunnels. It is possible to obtain an output image in which an object to be displayed is emphasized and an output image in which an object included in a bright portion is displayed to be emphasized.

By the way, when the image data of the low contrast area in the frame image is added for a plurality of frames by the above-described processing, the pixel values of all the pixels included in the low contrast area become the added number of frames. Therefore, the pixel value discontinuity occurs between the low contrast area and another area (area where image data addition processing is not performed) adjacent to the low contrast area, and at the boundary portion of these areas. In some cases, the image quality may be deteriorated such that the brightness changes extremely.

In such a case, the output image creation processing unit 6 adds the image data of the low contrast area for a plurality of frames so as to reduce the level difference of the pixel values between the adjacent areas, and then the low contrast area is added. It is desirable to correct the DC component of the image data in the contrast area so as to match it with another area adjacent to the low contrast area.

As a method of correcting the DC component of the image data in the low contrast area after the addition processing, for example,
A method is conceivable in which the spectrum intensity distribution for each frequency used in the above-described frequency analysis is reduced according to the number of frames to which the spectrum intensity indicating the DC component is added.

Explaining with a concrete example, for example,
As a result of frequency analysis of the low-contrast region in the frame image, if the intensity distribution of the spectrum intensity as shown in FIG. 6 is obtained, the image data of the low-contrast region is subjected to addition processing for, for example, 3 frames, and addition processing is performed. When frequency analysis is performed on the subsequent low contrast region, an intensity distribution of spectrum intensity as shown in FIG. 10 is obtained. As can be seen from FIG. 10, in the low-contrast region in the frame image, the spectrum intensity exceeding the threshold p0 can be obtained in the frequency band f0 to f0 ′ of interest by performing addition processing for three frames, The contrast is improved. However, in the low-contrast region after the addition processing, frequency components other than the frequency bands f0 to f0 'are also emphasized, and discontinuity of pixel values occurs between the low-contrast region and other regions adjacent to the low-contrast region. Is assumed.

Therefore, the frequency band of interest f0 to f0 '
The spectral intensities at frequencies other than the above are reduced in accordance with the number of added frames so that discontinuity of pixel values does not occur between the low contrast region after the addition process and the region adjacent thereto. In the above-described example, since the image data in the low contrast region is added for three frames, the spectral intensity at frequencies other than the frequency band f0 to f0 ′ of interest is 1 for the low contrast region after the addition process. / 3, so that the intensity distribution of the spectrum intensity as shown in FIG. 11 can be obtained. As a result, the DC component of the low-contrast region after the addition process is corrected, and the level difference in pixel value between the low-contrast region and other regions adjacent to this is relaxed.

As a method of reducing the level difference in pixel value between the low contrast area after the addition processing and another area adjacent to the low contrast area, a non-linear addition processing is performed between pixels in the low contrast area. In this way, the load when performing the addition process is maximized near the center of this low-contrast region and is minimized at the outer edge of this low-contrast region (boundary with other regions). It may be set to a value that changes dynamically. In this case, the difference in pixel value between the low contrast region and another region adjacent to the low contrast region is relaxed without performing the process of correcting the DC component in the low contrast region after the addition process. be able to.

(Second Embodiment) FIG. 12 is a block diagram showing another configuration example of the image processing apparatus to which the present invention is applied. The image processing apparatus 10 shown in FIG. 12 includes a CCD camera 11
A CCD camera, an A / D converter 12, a frame memory 13, an image analysis processing unit 14, a camera controller 15, an output image creation processing unit 16, and a D / A converter 17. By controlling the reset timing of the accumulated charge in 11 by the camera controller 15,
It is designed to improve the contrast in the low contrast area in the frame image.

That is, the image processing apparatus 1 of the above-described first embodiment realizes the contrast improvement in the low contrast area by adding the image data of the low contrast area in the frame image for a plurality of frames. On the other hand, in the image processing device 10 according to the second embodiment,
CCD corresponding to the low contrast area in the frame image
By delaying the reset timing of the accumulated charges in the imaging area of the camera 11 by a plurality of frames, the contrast in the low contrast area is improved.

In the image processing apparatus 10 of the second embodiment, the CCD camera 11 can pick up a frame image at a high frame rate, like the CCD camera 2 in the image processing apparatus 1 of the first embodiment described above. Is configured. Further, the CCD camera 11 can individually control the charge accumulation time in the image pickup section for each image pickup area of the image pickup section. This CCD camera 11
The voltage value signal from is supplied to the A / D converter 3.

The A / D converter 12 performs analog-digital conversion of the voltage value signal supplied from the CCD camera 11 as in the case of the A / D converter 3 in the image processing apparatus 1 of the first embodiment described above. Digital image data. The digital image data converted by the A / D converter 12 is supplied to the frame memory 13.

The frame memory 13 stores and holds the digital image data output from the A / D converter 12 in units of frames, like the frame memory 4 in the image processing apparatus 1 of the first embodiment. It is a thing.

The image analysis processing unit 14 is, like the image analysis processing unit 14 in the image processing apparatus 1 of the first embodiment, an image of a frame image captured by the CCD camera 11 and held in the frame memory 13. The frequency analysis is performed on the data to identify the low contrast area in this frame image. A specific method of identifying the low contrast region in the frame image by frequency analysis is the same as that of the first embodiment described above, and thus detailed description thereof is omitted here.

The camera controller 15 controls the reset timing of the accumulated charges accumulated in the image pickup section of the CCD camera 11 for each image pickup area of the image pickup section. Specifically, when the low-contrast area in the frame image is specified by the image analysis processing unit 14, the camera controller 15 stores the accumulated charge in the imaging area of the CCD camera 11 corresponding to the low-contrast area in the frame image. The control is performed to delay the reset timing of a plurality of frames. Here, the number of frames for delaying the reset timing of the accumulated charge, that is, the number of frames for delaying the reset timing of the accumulated charge in the image pickup area of the CCD camera 11 corresponding to the low contrast area in the frame image depends on the low contrast. It is determined according to the degree of low contrast in the area.

The output image creation processing section 16 creates the image data of the output image based on the image output from the CCD camera 11 under the control of the camera controller 15. That is, when the camera controller 15 controls to delay the reset timing of the accumulated charges in the image pickup area of the CCD camera 11 corresponding to the low contrast area by a plurality of frames, the CCD camera 11 outputs a plurality of voltage values in the low contrast area. The voltage value signal added for the frames is output. Then, when the image data according to the voltage value signal is supplied to the output image creation processing unit 16, the output image creation processing unit 16
Performs predetermined correction processing etc. on this image data,
Output as image data of output image.

The D / A converter 17, like the D / A converter 7 in the image processing apparatus 1 of the first embodiment described above, outputs the image data of the output image created by the output image creation processing unit 16. The digital-to-analog conversion is performed and output as an image signal. The image signal output from the D / A converter 17 is supplied to a display device such as a liquid crystal display. Then, the display device is driven based on this image signal, so that an image with improved contrast is displayed on the display device.

Here, the operation of the image processing apparatus 10 of the second embodiment configured as described above will be described.

When the image processing apparatus 10 is activated, the image of the object to be imaged (reflected light from the object) is transferred to the CCD.
An image is formed on the image pickup unit of the camera 11. Then, the reflected light from the subject is photoelectrically converted from photons into electric charges in the imaging unit of the CCD camera 11, and the electric charges accumulated during the electric charge accumulation time of the CCD camera 11 are converted into a voltage to form a frame image. Is output as a voltage value signal indicating the voltage value (pixel value) of each pixel.

Here, normally, the charge accumulated in the image pickup section of the CCD camera 11 is reset every time one frame image is picked up, and when the next frame image is picked up,
In the state where there is no residual electric charge, the electric charge is newly accumulated according to the reflected light of the subject. On the other hand, in the image processing apparatus 10 of the second embodiment to which the present invention is applied, the CCD controller 1 is controlled by the camera controller 15.
The reset timing of the accumulated charge accumulated in one image pickup unit is controlled for each image pickup region of this image pickup unit, and the reset timing of the specific image pickup region of the image pickup unit of the CCD camera 11 The charge corresponding to the new frame image can be added while the charge corresponding to the frame image remains.

For simplicity, one pixel in the frame image will be described. Normally, as shown in FIG. 13A, the first frame image is captured at time t0,
Assuming that the charge Q0 is accumulated within the charge accumulation time Δt0, the voltage value (pixel value) of this pixel is output as a value corresponding to the charge Q0 in the first frame image. Then, if the imaging of the next frame image is started at time t1, the charge Q0 accumulated within the charge accumulation time Δt0 of the first frame image is reset before time t1. Then, at time t1, the image pickup of the next frame image is started, and if the charge Q1 is accumulated within the charge accumulation time Δt1, the voltage value (pixel value) of this pixel corresponds to the charge Q1 in the next frame image. Is output as the value.

On the other hand, in the image processing apparatus 10 of the second embodiment to which the present invention is applied, as shown in FIG. 13B, the image pickup of the first frame image is started at time t0, and the charge accumulation is started. When the charge Q0 is accumulated within the time Δt0,
In the first frame image, the voltage value (pixel value) of this pixel
Is output as a value corresponding to the charge Q0. Then, when the camera controller 15 controls to delay the reset timing of the accumulated charge corresponding to this pixel, the charge Q0 accumulated in the charge accumulation time Δt0 of the first frame image is not reset, and t1
At, the capturing of the next frame image is started. Then, when the electric charge ΔQ1 is further accumulated within the electric charge accumulation time Δt1 by adding it to the remaining electric charge Q0, the voltage value (pixel value) of this pixel is changed to the electric charge QQ in the next frame image.
It is output as a value corresponding to 0 + ΔQ1.

The process of adding the accumulated charges of the CCD camera 11 as described above has the same meaning as the process of adding the image data in the image processing apparatus 1 of the first embodiment described above. That is, in the image processing apparatus 1 according to the above-described first embodiment, the output image creation processing unit 6 performs a process of adding image data for a plurality of frames as a method of enlarging the pixel value difference in the low contrast region. In the image processing apparatus 10 of the second embodiment, the camera controller 15 controls the reset timing of the accumulated charges in the CCD camera 11 to perform the process of adding the accumulated charges of the CCD 11.

The voltage value signal output from the CCD camera 11 is converted into digital image data representing a frame image by the A / D converter 12 and stored in the frame memory 13. Then, the image data of the frame image held in the frame memory 13 is taken into the image analysis processing unit 16. Similar to the image analysis processing unit 5 in the image processing apparatus 1 according to the first embodiment described above, the image analysis processing unit 16 performs a frequency analysis on the frame image captured from the frame memory 13 and displays the analysis result. Originally, the low contrast area in this frame image is specified.

When the image analysis processing unit 16 frequency-analyzes the frame image to identify the low contrast area in the frame image, the camera controller 15 continues until the contrast of the low contrast area is sufficiently improved. Processing for delaying the reset timing of the accumulated charges in the image pickup area of the CCD camera 11 corresponding to the low contrast area is performed.

That is, a plurality of frame images (frame image group) captured within one output cycle of the output image.
When the low-contrast region in the first frame image is specified by the image analysis processing unit 16 performing frequency analysis on the first frame image of the, the camera controller 15 causes the camera controller 15 to capture an image of the low-contrast region of the CCD camera 11. The reset timing of the accumulated charges in this region is delayed so that the accumulated charges in the area are retained until the next frame image is captured. This allows the CCD camera 1
The next frame image captured by 1 will be output in a state in which the pixel values in the low contrast region are added.

Then, the image analysis processing unit 16 performs frequency analysis on this frame image, and when it is determined that the contrast of the low contrast region in this frame image is not sufficiently improved, the camera controller 1
Reference numeral 5 further delays the reset timing of the accumulated charge in this region so that the accumulated charge in the image pickup region of the CCD camera 11 corresponding to this low contrast region is held until the time of capturing the next frame image.

The above-described processing is repeated until the contrast in the low contrast area is sufficiently improved. Then, the frame image in which the contrast of the low contrast region is sufficiently improved is output image creation processing unit 16
Will be taken into account.

When the frame image in which the contrast in the low contrast region is sufficiently improved is supplied, the output image creation processing unit 16 adds a low contrast region in which the contrast is improved to this frame image and the adjacent low image region. Pixel value correction processing for alleviating the discontinuity of pixel values between the regions to be processed is performed. Then, the frame image subjected to such pixel value correction processing is output as an output image. The correction process for alleviating the discontinuity of the pixel values is performed by continuously changing the charge accumulation time in the image pickup area of the CCD camera 11 corresponding to the low contrast area in the frame image in the image pickup area and adjoining the charge accumulation time. A process of matching the pixel values at the boundary with the region may be performed.

According to the image processing apparatus 10 of the second embodiment described above, the camera controller 15 causes the CC
Since the reset timing of the accumulated charge in the D camera 11 is controlled so that the contrast in the low contrast region in the frame image is improved, as in the image processing apparatus 1 of the first embodiment described above, It is possible to obtain an output image in which the background and the background can be clearly distinguished.

In addition, in the image processing apparatus 10 of the second embodiment, the accumulated charge in the image pickup area of the CCD 11 corresponding to the low contrast area is added to enlarge the pixel value difference in the low contrast area to increase the contrast. Since the improvement is made, it is possible to obtain a more reliable output image as compared with the case where the contrast in the low contrast area is improved by digital signal processing such as addition of image data.

In the embodiment described above, C is used as the image pickup means for picking up the image of the subject as a frame image.
Although the CD cameras 2 and 11 are used, if the image of the subject can be captured at a high frame rate,
In addition to the CCD cameras 2 and 11, other image pickup means such as a CMOS camera may be used. Also in this case, the same effect as when the CCD cameras 2 and 11 are used can be obtained.

Further, in the above-mentioned embodiments, description has been made on the assumption that the output image which is output with the contrast improved by the image processing apparatuses 1 and 10 is displayed on the display means, but the output with the improved contrast is described. The image is very effective when used as an image for detecting, for example, a preceding vehicle or an obstacle, in addition to being displayed on the display means. That is, in such an output image with improved contrast, it is possible to clearly distinguish the object from the background, and therefore it is possible to detect the preceding vehicle, obstacles, etc. with high accuracy by using such an output image. Becomes

[Brief description of drawings]

FIG. 1 is a block diagram showing an image processing apparatus of a first embodiment to which the present invention is applied.

FIG. 2 is a diagram showing an example of a frame image captured by a CCD camera included in the image processing apparatus.

FIG. 3 is a diagram illustrating a relationship between an image and a spatial frequency, (a) showing an unclearly captured original image, and (b).
Shows a Fourier transform image obtained by Fourier transforming the original image of (a).

FIG. 4 is a diagram illustrating a relationship between an image and a spatial frequency, where (a) shows an original image captured to some extent clearly,
(B) shows a Fourier transform image obtained by Fourier transforming the original image of (a).

5 is a diagram showing a Fourier transform image obtained by Fourier transforming a region A of the frame image shown in FIG.

6 is a diagram showing an intensity distribution of spectral intensities for each frequency of the Fourier transform image shown in FIG.

FIG. 7 is a schematic diagram illustrating a process of creating an output image by performing an addition process on image data in a low contrast area of a frame image.

FIG. 8 is a flowchart showing a processing flow in an image analysis processing unit and an output image creation processing unit which realize improvement of image contrast.

9 is a diagram showing an output image obtained as a result of performing addition processing of image data on a region A of the frame image shown in FIG.

FIG. 10 is a diagram showing an intensity distribution of spectral intensities for each frequency in a low contrast region in which image data addition processing has been performed.

FIG. 11 is a diagram illustrating an example of correcting a DC component in a low contrast region where image data addition processing has been performed.

FIG. 12 is a block diagram showing an image processing apparatus of a second embodiment to which the present invention is applied.

FIG. 13 is a diagram illustrating an accumulated charge amount for each frame image captured by a CCD camera, in which FIG.
The accumulated charge amount corresponding to one pixel in the frame image captured by the CD camera is shown, and (b) corresponds to one pixel in the low contrast region of the frame image captured by the CCD camera included in the image processing apparatus. The amount of accumulated charge is shown.

[Explanation of symbols]

1 Image processing device 2 CCD camera 5 Image analysis processor 6 Output image creation processing unit 10 Image processing device 11 CCD camera 14 Image analysis processing unit 15 camera controller 16 Output image creation processing unit

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5B057 BA02 CA08 CA12 CA16 CB08                       CB12 CB16 CC01 CE08 CE11                       CH08 CH11                 5C022 AC42 AC69                 5C023 AA37 BA01 BA08 CA01 DA04                 5C024 AX01 BX01 CX65 DX04 GY01                       HX23 HX58

Claims (7)

[Claims] 1. An image pickup means for picking up a frame image in a cycle shorter than the output cycle of an output image, a frequency analysis of the frame image picked up by the image pickup means, and based on the analysis result, A frequency analysis unit that specifies a low contrast region, and an output image creation unit that creates the output image by adding a plurality of frames of image data of the region specified as the low contrast region by the frequency analysis unit. An image processing device characterized by the above. 2. The frequency analysis unit specifies, in the frame image output from the image pickup unit, a region in which a spectral intensity exceeding a predetermined threshold value is not obtained in a predetermined frequency band as a low contrast region. The image processing device according to claim 1. 3. The output image creating means adds the image data of the low contrast area for a plurality of frames and then performs a correction process to bring the DC component of the image data closer to the DC component of the image data of the adjacent area. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. 4. The output image creating means calculates a correction amount of the correction process based on a spectrum intensity indicating a DC component of image data in the low contrast area. Image processing device. 5. An image pickup means capable of picking up a frame image at a cycle shorter than the output cycle of the output image, frequency analysis of the frame image picked up by the image pickup means, and based on the analysis result, the frame image And a storage control for delaying the reset timing of the stored charges in the image pickup region of the image pickup unit corresponding to the region specified as the low contrast region by the frequency analysis unit by a plurality of frames. An image processing apparatus comprising: a means and an output image creation means for creating the output image based on an image output from the imaging means under the control of the storage control means. 6. The frame image is acquired from an image capturing unit that captures the frame image at a cycle shorter than the output cycle of the output image, the acquired frame image is subjected to frequency analysis, and the frame image is obtained based on the analysis result. A low-contrast region in the inside is specified, image data of the low-contrast region in the frame image is added for a plurality of frames, and the output image is created. 7. The frame image is acquired from an imaging means capable of capturing the frame image at a cycle shorter than the output cycle of the output image, the acquired frame image is subjected to frequency analysis, and the frame is obtained based on the analysis result. A region with low contrast in the image is specified, and control is performed to delay the reset timing of the accumulated charges in the image pickup region of the image pickup unit corresponding to the region with low contrast in the frame image by a plurality of frames,
An image processing method, wherein the output image is created based on an image output from the image pickup means in accordance with this control.