JP2003087644A - Device and method for picking up and displaying image and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup and display device which can display an easy-to-see photographed image at a display part by canceling the influence of disturbing light even if a display object area is irradiated with the diffused external light. SOLUTION: The circumference of a vehicle is irradiated with near infrared light modulated by a light quantity modulation part 2 from an irradiation part 1 provided to the vehicle in addition to ordinary headlights. Images are picked up at a point (p) of time when the quantity of the irradiating light is large and a point (q) of time when the quantity is small and the obtained images are recorded in an image recording part 4. An image difference part 5 finds differences between the two I images to cancel the influence of the disturbing light. A luminance correction part 7 corrects the luminance of the difference image by multiplying the luminance of the image of differences by a specific amplification factor. The corrected difference image is displayed at a display part 7 provided facing a driver in the vehicle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup display device, an image pickup display method, and a program for displaying an obtained image by irradiating light to pick up an image.

[0002]

2. Description of the Related Art Conventionally, as an image display device of this type, for example, it is mounted on a vehicle and emits near-infrared light in the middle of night when visibility is poor, and an image of the surroundings of the vehicle is imaged to a driver in the vehicle. There is known a vehicle peripheral display device for displaying it.

FIG. 8 is a view showing the appearance of a vehicle equipped with a conventional vehicle peripheral display device. Generally, a vehicle is equipped with a normal visible light (white light) headlight 120 in order to enhance visibility at night, but there is also an influence on an oncoming vehicle, so a wide field of view (far ahead). The irradiation range cannot be expanded to the extent that In other words, ordinary headlights are often used with a low beam instead of a high beam. In that case, a wide field of view cannot be secured.

On the other hand, an irradiating section 101 for irradiating only near-infrared light, which is invisible to humans, through an optical filter.
The image pickup unit 10 which irradiates this light toward a distance like a high beam and senses not only visible light but also near infrared light.
A vehicle peripheral display device has been proposed which provides a driver with an image in a visual field range equivalent to that of a high beam that does not dazzle an oncoming vehicle by capturing the image in 3 above.

For example, the vehicle periphery display device described in Japanese Patent Laid-Open No. 2000-115759 is realized by mounting an image pickup display device capable of photographing even at night on the vehicle. FIG. 9 is a diagram showing a configuration example of a conventional vehicle peripheral display device. The irradiation unit 101 mainly irradiates light in the near infrared region to the imaging target region. When the illumination unit 101 is used in a state where sufficient illuminance can be obtained, such as in the daytime, the irradiation unit 101 does not have to be used, but in a state where the illuminance is not sufficient such as at night, illumination by the irradiation unit 101 is performed.

The image pickup unit 103 is composed of an infrared camera, and picks up an image of the area illuminated by the irradiation unit 101. The display unit 107 is, for example, a liquid crystal display used in a car navigation system. Part 1
The image captured by 03 is displayed toward the driver in the vehicle.

As described above, normally, even when the illuminance is insufficient and a sufficiently clear photographed image cannot be obtained, the irradiation unit 101
Therefore, the illuminance can be secured and a clear image can be provided to the driver.

[0008]

However, in the above-mentioned conventional vehicle peripheral display device, in addition to the light emitted by itself,
Light from different light sources (for example, street lights or lights of oncoming vehicles, hereinafter referred to as disturbance light) may be mixed, and the influence of the disturbance light overlaps a part of the image and may be difficult to see, or it may be difficult to see. There is a possibility that the image becomes difficult and the visibility of the driver is reduced.

The present invention has been made in view of the above circumstances, and even when ambient light is applied to the display target area, the effect of the ambient light can be canceled and an easily viewable captured image can be displayed on the display unit. An object is to provide an image pickup display device, an image pickup display method, and a program that can be performed.

[0010]

An image pickup display apparatus according to the present invention comprises a light irradiating means for irradiating a display target area with light, and a light quantity modulating means for temporally changing the light quantity of the light irradiating means.
An image pickup means for picking up an image of a display target area irradiated by the light irradiation means, an image difference means for outputting a difference image based on a plurality of images picked up by the image pickup means, and the image difference means Display means for displaying the output difference image, wherein the image difference means outputs the difference image based on a plurality of images captured at different timings of the light irradiation means. is there. With the above configuration, even if ambient light is applied to the display target area,
The influence of the ambient light can be canceled by the difference image, and the photographed image that is easy to see can be displayed on the display unit.

Further, in the image pickup display apparatus according to the present invention, the light quantity modulating means controls the light irradiating means so as to periodically repeat lighting and extinguishing, and the imaging means controls the lighting and extinguishing respectively. The image is taken at a timing. In the above configuration, the influence of the ambient light is canceled by capturing the image at each timing of turning on and off and generating the image of the difference.

Further, in the image pickup display device of the present invention, the light quantity modulating means controls the light irradiating means so as to repeat the light and dark continuously and periodically, and the image pickup means is arranged in the vicinity of the respective peaks of the light and dark. Is to capture the image. In the above configuration, the influence of ambient light is canceled by capturing an image near each of the light and dark peaks and generating a difference image.

Further, the image pickup display device of the present invention comprises a brightness correction means for correcting the brightness of the difference image outputted by the image difference means, and the brightness correction means is required by the display means. The standard brightness is a, the average brightness of the image taken when the light quantity of the light irradiation means is large among the plurality of images taken by the imaging means is b, and the average brightness is taken when the light quantity of the light irradiation means is low. When the average brightness of the image is c, the brightness is corrected with a / (bc) as the amplification factor. With the above configuration, the brightness of the image can be adjusted so that the entire screen is not darkened by correcting the brightness with a predetermined amplification factor.

Further, the image pickup display device of the present invention comprises an output image judging means for comparing the image quality of the raw image picked up when the light irradiation means has a large amount of light and the image of the difference outputted by the image difference means. The output image determining means compares the image quality based on the sharpness of the edge of the object shown in the image, and outputs the image with the higher sharpness to the display means. According to the above configuration, it is possible to compare the image quality based on the sharpness and output the raw image itself when there is a possibility that a difference may cause an adverse effect, thereby always obtaining a clear image.

Further, the image pickup display device of the present invention comprises an image recording means for recording a raw image taken when the light quantity of the light quantity modulating means is large, and an average brightness calculating means for calculating an average brightness of the difference image. A mixing ratio calculation unit that calculates a mixing ratio when the raw image and the difference image are mixed to generate one image based on the average brightness calculated by the average brightness calculation unit; Based on the mixture ratio calculated by the mixture ratio calculation unit, a brightness correction unit that corrects the brightness of each of the two images including the raw image and the difference image, and the brightness corrected by the brightness correction unit 2 An image mixing unit that adds and mixes the images on a pixel-by-pixel basis, and the display unit displays the mixed image. According to the above configuration, by weighting and mixing the raw image and the difference image, it is possible to obtain a clear display image in which the influence of ambient light is removed when the surroundings are dark such as at night. At the same time, it is possible to obtain a sufficiently recognizable display image even when it is bright in the daytime, when it is cloudy or in the morning when it is relatively dark.

Further, the image pickup display device of the present invention includes a light irradiation means for irradiating the display target area with light, a light quantity modulation means for outputting modulation information for temporally changing the light quantity of the light irradiation means, and the light. Image pickup means for picking up the image of the display target area irradiated by the irradiation means, and multiplication for multiplying the modulation information output from the light quantity modulation means at the timing of the image pickup by the image pickup means and the brightness information of the picked-up image. Means, an integrating means for integrating the result of the multiplication by the multiplying means over a predetermined time, and a result of the integration by the integrating means,
It is provided with a brightness correction means for correcting the brightness of the obtained image and a display means for displaying the image of which the brightness is corrected by the brightness correction means. According to the above configuration, for a predetermined time, for example, a time equal to or longer than the modulation cycle of the light irradiation means,
By integrating, it is possible to remove other disturbance components and obtain a clear display image.

The image pickup display device of the present invention is mounted on a vehicle, and the situation around the vehicle is used as the display target area. By applying it to the vehicle-mounted device as described above, the driver can obtain a clear image of the surroundings of the vehicle, for example, even when the forward view is dark, and the visibility can be secured well.

The image pickup display method of the present invention comprises a light irradiation step of irradiating the display target area with light while changing the light amount,
An image capturing step of capturing a plurality of images of the display target area irradiated in the light irradiation step, and an image difference step of outputting a difference image based on the plurality of captured images,
A display step of displaying the image of the difference output in the image difference step, wherein the image difference step,
A difference image is output based on a plurality of images captured at different timings in the light irradiation step. With the above configuration, even if the ambient light is applied to the display target area, the influence of the ambient light can be canceled by the difference image, and it is possible to display a photographed image that is easy to see on the display unit.

Further, according to the image pickup display method of the present invention, a light irradiation step of outputting modulation information for temporally changing the light quantity when the light is irradiated to the display target area, and the light is irradiated in the light irradiation step. The image capturing step of capturing an image of the display target region, the multiplication step of multiplying the modulation information output at the image capturing timing in the image capturing step by the brightness information of the captured image, and the multiplication step. An integration step of integrating the multiplication result over a predetermined time, a brightness correction step of correcting the brightness of an image obtained as a result of the integration obtained in the integration step, and an image corrected in the brightness correction step are displayed. And a display step to perform. According to the above configuration, it is possible to remove components due to other disturbances by integrating over a predetermined time, for example, for a time equal to or longer than the modulation cycle of the light irradiation means, and it is possible to obtain a clear display image. Becomes

The present invention also provides a program for causing a computer to execute the above-described image pickup display method.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. In the present embodiment, as an example of the image pickup display device, a configuration that is mounted on a vehicle and is applied to a vehicle periphery display device that displays a situation around the vehicle is shown.

(First Embodiment) FIG. 1 is a block diagram showing a configuration of a vehicle peripheral display device according to a first embodiment of the present invention. This vehicle periphery display device includes an irradiation unit 1, a light amount modulation unit 2, an imaging unit 3, an image recording unit 4, an image difference unit 5, a brightness correction unit 6, and a display unit 7. Irradiation unit 1
Is provided in a vehicle in addition to a normal headlight, and irradiates the display target area mainly with light in the near infrared region, which is outside the visible light region. When the vehicle peripheral device is used in a daytime state in which sufficient illuminance can be obtained, it is not necessary to use the irradiation unit 1, but in a state such as at night when the illuminance is not sufficient, illumination by the irradiation unit 1 is performed. Then, the imaging unit 3 images the area illuminated by the irradiation unit 1.

In this embodiment, the light quantity of the light emitted by the irradiation section 1 is temporally changed according to the modulation signal of the light quantity modulation section 2. FIG. 2 is a graph showing a temporal change of the modulation signal of the light quantity modulator 2. In the figure, the vertical axis represents the light quantity or a parameter proportional to the light quantity, and the horizontal axis represents time.

As shown in FIG. 2A, the modulation signal may be a signal which changes in a rectangular shape between a certain constant light amount d [Watt] and zero light amount, or FIG. 2B. As shown in, the signal may be a signal that sinusoidally changes between the light amount d [Watt] and the light amount e [Watt]. As will be described later, since image processing is performed on adjacent bright and dark images, it is preferable that the modulation cycle is as short as possible in order to cope with external dynamic changes. For example, 3 which is the frame update period of the image
By operating at a cycle of 0 msec, or 15 msec which is half of that, it is possible to sufficiently cope with external dynamic changes.

Further, as shown in FIG. 2 (A), the irradiation unit 1
It may be difficult to completely switch on / off the illumination by the method when the responsiveness of the irradiation unit 1 is poor. In this case, it is effective to perform sinusoidal modulation as shown in FIG.

The image recording section 4 is means for recording the image data picked up by the image pickup section 3 on a pixel-by-pixel basis.
Images are recorded both at the time of the highest illuminance (light time) and at the time of the lowest illuminance (dark time). For example, in the case of FIGS. 2A and 2B, each image acquired at two time points p and q is recorded. The image difference unit 5 acquires the difference between the two images recorded in the image recording unit 4. When acquiring the difference of this image, the difference of the brightness information of the image imaged at each timing of light and dark is acquired for each pixel. Also,
As the difference of the brightness information, the difference of the intensity information of each color such as RGB constituting the display color may be acquired.

Here, as shown in FIG. 1, both the irradiation light from the irradiation unit 1 and the disturbance light are mixed in the display target area,
Although the two images captured as described above have a difference in brightness and darkness due to the irradiation unit 1, as shown in FIGS. 3A and 3B, the influences of the ambient light a1 and a2 are mixed almost equally. Will be done. FIG. 3 is an operation explanatory view showing a captured raw image and a processed image subjected to image processing. Figure 3
Is an image explanatory view of the front of the driver's seat of the automobile captured by a camera, showing the road surface in the traveling direction, and the influence of disturbance light incident on the camera as disturbance is shown as oval noise in the figure. The figure shows that it is mixed in several places. However, it is assumed that the temporal difference of ambient light with respect to the light quantity modulation period can be ignored. In the image of the difference obtained by the image difference unit 5, the influence of ambient light is removed on a pixel-by-pixel basis (see FIG. 3C).

The brightness correction unit 6 corrects the difference image obtained by the image difference unit 5 by multiplying the brightness of each pixel by a constant amplification factor (generally 1 or more) (FIG. 3 (D)). reference).
As will be described later, the modulation signal of the light quantity modulator 2 is shown in FIG.
This correction is not always necessary for the signal shown in FIG. 2, but if it is modulated with a sine wave and the irradiation unit 1 is not completely turned off at the dark point q as shown in FIG. By taking the difference in the image difference unit 5, the dynamic range is determined by the amplitude of the modulation, and the brightness of the entire screen may be reduced and the screen may become dark. Therefore, the brightness of the entire screen is adjusted by the brightness correction as described above.

The display unit 7 is, for example, a liquid crystal display used in a car navigation system,
The captured image is displayed to the driver in the car.

Here, the light quantity modulating unit 2, the image recording unit 4, the image difference unit 5 and the brightness correcting unit 6 are a well-known CPU,
It is realized by using a computer unit (not shown) having a ROM and a RAM.

An image pickup display process of the vehicle periphery display device having the above configuration will be described. FIG. 4 is a flowchart showing the procedure of image pickup display processing. This processing program is stored in the ROM in the computer unit described above,
Similarly, it is repeatedly executed by the CPU in the computer unit.

First, in addition to the normal headlight, near-infrared light modulated by the light quantity modulator 2 from the irradiation unit 1 provided in the vehicle is applied to the periphery of the vehicle (step S1). Then, when the amount of the emitted light is large (light time) p,
The image is captured by the image capturing unit 3 at a time point when the amount of light is low (dark time point) q, and the obtained image is recorded (step S
2). Next, the image difference unit 5 takes the difference between the two images to cancel the influence of ambient light (step S3).
Further, the brightness of the difference image is corrected by multiplying the brightness of the difference image by a predetermined amplification factor (step S4). Then, the corrected difference image is displayed on the display unit 7 provided for the driver in the vehicle (step S5). After this,
The process ends.

As described above, in the vehicle periphery display device of the present embodiment, even when the illuminance is usually insufficient and a sufficiently clear photographed image cannot be obtained, the illuminator 1 ensures the illuminance and the disturbance. It removes or reduces the influence of light and provides the driver with a clear image that is not affected by ambient light.

Here, the difference in the modulation signal output from the light quantity modulator 2 will be supplemented. As shown in FIG. 2A, when the modulation signal of the light quantity modulator 2 is completely on / off switched in a constant cycle, the illumination by the irradiation unit 1 is also completely on / off.

As long as the responsiveness of the irradiation section 1 is high, the switching of the light quantity modulation section 2 which performs such an operation can be realized by a simple circuit. For example, the light quantity modulator 2 may output a pulse train that repeats on / off at a cycle of 30 msec which is a frame update cycle of an image, or 15 msec which is a half thereof. The irradiation unit 1 repeats blinking according to the pulse signal. The image difference unit 5 calculates a difference image from two adjacent bright and dark frame images. At this time, since the illumination by the irradiation unit 1 is completely zero on the dark screen, the dynamic range is not reduced by the difference, and the brightness correction unit 6 is not necessary.
Alternatively, there is an advantage that a slight gain correction is sufficient.

On the other hand, as shown in FIG. 2B, when the modulation signal of the light quantity modulator 2 is a sine wave, the illumination by the irradiation unit 1 is modulated with a sine wave. The sine wave itself can be easily generated by preparing a simple table (look-up table (reference table) recorded in a ROM or the like). In addition, it is possible to avoid damage to the irradiation unit 1 by repeating blinking of the illumination by blinking by switching. The image capturing unit 3 captures images near the bright and dark peaks generated in the irradiating unit 1, and acquires images at the bright time point and the dark time point.

Even if the light quantity modulator 2 outputs a pulse-shaped modulation signal, if the time response of the irradiation unit 1 is poor, the waveform is blunted and the light quantity modulator 2 increases or decreases the amplitude. The operation is equivalent to outputting a repetitive wavy signal. In this way, even if the lighting unit cannot be completely turned off by switching for some reason, the function of the vehicle peripheral display device can be realized.

Further, as described above, the modulation signal of the light quantity modulator 2 is modulated with a sine wave as shown in FIG.
In addition, when it is not completely turned off in the dark part, the difference between the two images is obtained by the image difference part 5, so that the dynamic range is determined by the amplitude of the modulation as it is, and the brightness of the entire screen is reduced. , It may get dark.

Therefore, the brightness correction unit 6 sets the standard brightness required for the final output display image to a [cd / m ² ], and the average brightness of the image acquired at the bright timing to b [cd / m ² ].
² ], the average brightness of the image acquired at the dark timing is c
When [cd / m ² ] is set, the luminance of the image obtained by the image difference unit 5 is defined as a / (bc), which is the ratio of the luminance component dropped by the difference image and the luminance component desired to be obtained. Is multiplied by a constant amplification factor (generally, a value of 1 or more) for each pixel to correct the luminance.

For example, when it is desired to obtain an image when the irradiation unit 1 is illuminated most brightly, a [cd / m ² ] and b [cd /
m ² ] are made equal and the amplification factor is set to b / (bc), whereby an image having a desired brightness can be obtained. This makes it possible to obtain an image that is the same in brightness as the original image even after the influence of ambient light is removed.

(Second Embodiment) FIG. 5 is a view showing the arrangement of a vehicle periphery display device according to the second embodiment. In the second embodiment, as compared with the first embodiment, the output image determination unit 8
Has been added. Other configurations are the same as those in the first embodiment, and the same reference numerals are given, and the description thereof is omitted.

That is, in the vehicle periphery display device according to the second embodiment, an image acquired at a bright timing among bright and dark, that is, a raw image when the irradiation unit 1 is illuminated most brightly,
It is displayed as a display image of final output.

For example, if there is a very strong light source such as daylight and the illuminance of the light is much larger than that of the irradiation unit 1, the influence of the light source is almost dominant to the image, and the image is bright and dark. If the difference is taken, it is assumed that most of the image information will disappear.

However, in this case, since there is a strong light source,
It should be possible to display the raw image as it is. Therefore, the output image determination unit 8 reflects the difference image obtained through the image difference unit 5 and the brightness correction unit 6 and the raw image at the bright timing recorded in the image recording unit 4 in the image. The sharpness of the edge of the object is automatically compared, and the image with the sharper edge is output to the display unit 7 as an image with high image quality.

As described above, in the second embodiment, when there is a possibility that the difference may cause an adverse effect, a clear image can be always obtained by outputting the raw image itself. In this case, the function of the imaging display processing program is as shown in FIG.
After the brightness correction processing shown in step S4 of
It is realized by adding an output image determination process as A.

(Third Embodiment) FIG. 6 is a view showing the arrangement of a vehicle periphery display device according to the third embodiment. The irradiation unit 1, the light quantity modulation unit 2, the image pickup unit 3, the image recording unit 4, the image difference unit 5, and the display unit 7 in the third embodiment are the same as those in the first embodiment, and therefore their explanations are omitted. However, here, only components different from the first embodiment will be described.

First, the image obtained by the image difference unit 5 will be supplemented. The image of the difference obtained by the image difference unit 5 is the difference between the images obtained at the respective timings of light and dark,
When the illumination of the irradiation unit 1 is dominant, the image is a very clear image of the surrounding situation. But if sunlight is dominant, for example during the day on a clear day,
If the difference between the bright and dark images is taken, there is a risk that the image will be a little dark, for example, a completely dark image. In this case, as in the case of the second embodiment, even in a raw image, the surrounding situation can be clearly seen by sunlight, and for example, an image taken at a bright timing should be sufficiently clear. It is enough to display the raw image, but on a cloudy day or the morning when the sunlight is not so strong, the raw image acquired at the timing of the light may not be enough to see the surrounding situation. .

Therefore, in the third embodiment, the surrounding situation is made visible by using both the raw bright image and the bright / dark difference image. For this reason, in the third embodiment, the average luminance calculation unit 11, the weighting calculation unit 12, the image mixing unit 13, and the luminance correction units 6A and 6B are provided. Average brightness calculator 1
1 calculates the average brightness which is the average of the brightness of each pixel in the difference image. When the average brightness is high, it indicates that a clear difference image is obtained, while when the average brightness is low, the difference image is expected to be dark and not clear.

The weighting calculation section 12 is the average luminance calculation section 1
Based on the calculation result of 1, each mixing ratio (weighting) used when one image is generated by combining the image of the bright timing and the difference image is calculated. For example, when the average brightness is high, the difference image is clearly obtained, so the weighting of the difference image is increased. Conversely, when the average brightness is low, the weighting of the image at the latest bright timing recorded in the image recording unit 4 is performed. Try to increase.

The brightness correction units 6A and 6B set the brightness of each pixel in the bright image stored in the image recording unit 4 and the difference image obtained by the image difference unit 5 to a constant value according to the above weighting. The amplification factor is multiplied to correct the brightness of each image. In addition, RGB of each pixel as the brightness information of each pixel
Weighting may be applied to each value.

The pixel mixing unit 13 adds the corrected difference image that has been appropriately weighted through the brightness correction unit 6B and the bright raw image that has been appropriately weighted through the brightness correction unit 6A to form a mixed image. To generate. In addition, RGB of each pixel
The values may be added to each other to form a mixed image.

The average brightness calculated by the average brightness calculating unit 11 may be the entire screen, or if the clarity of the screen is expected to be different, the screen may be divided into several blocks. Alternatively, each block may be divided. In this case, weighting may be calculated for each block, luminance correction may be performed, and addition may be performed to form a mixed image.

In the above embodiment, a bright raw image is used, but when sunlight is dominant, a raw raw image acquired at a dark timing may be used instead of the bright raw image.

As a result, it is possible to obtain a clear display image in which the influence of ambient light such as the headlight of another vehicle is removed at night, and at the same time, during the daytime when the sunlight is dominant or when the sunlight is not so much. It is possible to obtain a display image that can be sufficiently recognized by the driver even on a cloudy day or in the morning, which is not dominant.

In this case, the function of the image pickup display processing program is that, after the image difference processing shown in step S3 of FIG. 4, the average luminance calculation processing is performed in step S3B and the weighting processing is performed in step S3C, and instead of step S4. This is realized by performing the brightness correction processing of the difference image / raw image in step S4A and the image mixing processing in step S4B.

(Fourth Embodiment) FIG. 7 is a view showing the arrangement of a vehicle periphery display device according to the fourth embodiment. The irradiation unit 1, the light quantity modulation unit 2, the image pickup unit 3, the brightness correction unit 6, and the display unit 7 in the fourth embodiment are the same as those in the first embodiment, and therefore the description thereof will be omitted. The first
Only components different from the embodiment will be described.

That is, in the fourth embodiment, the multiplication unit 9 and the integration unit 10 are added. The multiplication unit 9 includes the imaging unit 3
The modulation signal output from the light intensity modulation unit 2 at the timing of image capturing by and the luminance signal of the captured image are multiplied in pixel units. Note that intensity information of each color such as RGB that constitutes a display color may be used as the brightness information, and the intensity information and the modulation information may be multiplied in pixel units.

Further, the integrating section 10 integrates the output signal as the multiplication result output from the multiplying section 9 on a pixel-by-pixel basis over a predetermined time. The integration of the predetermined time may be realized by, for example, adding all the output signals from the multiplication unit 9 for a period corresponding to the modulation period of the light amount modulation unit 2. Also,
This multiplication / integration operation corresponds to obtaining the correlation between the output signal from the multiplication unit 9 and the modulation signal output from the light amount modulation unit 2, and removes the component due to other disturbances by the integration of the modulation period or more. be able to.

As described above, the modulation signal of the light quantity modulator 2 and the luminance signal of each pixel of the captured image are multiplied by the multiplier 9, and the output signal of the multiplier 9 is at least modulated by the integrator 10 at the modulation cycle. By integrating for the above time, the signal of the DC component due to ambient light or the like is almost removed, and only the signal of the component in which the light amount is modulated remains.

This makes it possible to remove the influence of ambient light and obtain a clear display image. In this case,
The function of the image pickup display processing program is as shown in step S2 of FIG.
It is realized by performing the image capturing / multiplying process in step S2A instead of the image capturing / recording process shown in FIG. 6 and performing the integration process in step S3D instead of the image difference calculation process shown in step S3.

The above is a description of the embodiments of the present invention, but the present invention is not limited to the configurations of these embodiments, and the functions shown in the claims or the configurations of the embodiments are not limited thereto. Any structure can be applied as long as it can achieve the function it has.

For example, in the above embodiment, the case of being mounted on a vehicle has been shown, but the present invention is not limited to this, and can be applied to a security camera of a building or the like, a visual sensor of a traveling robot, and the like.

Further, when the present invention is achieved by supplying the apparatus with the program code of the software that realizes the functions of the above-described embodiments, the program code itself realizes the novel function of the present invention. The program itself and the storage medium storing the program constitute the present invention.

In the above embodiment, the program code shown in the flowchart of FIG. 4 is stored in the storage medium.
As the storage medium for supplying the program code, for example, a ROM, a flexible disk, a hard disk,
A non-volatile memory card or the like can be used.

[0065]

As described above, according to the present invention, even when the ambient light is applied to the display target area, it is possible to cancel the influence of the ambient light and display a photographed image that is easy to see on the display unit. It is possible to obtain the effect of providing a different image display device, image display method, and program.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a vehicle peripheral display device according to a first embodiment of the present invention.

FIG. 2 is a graph showing a temporal change of a modulation signal of a light quantity modulator in the first embodiment.

FIG. 3 is an operation explanatory diagram showing a captured raw image and a processed image subjected to image processing in the first embodiment.

FIG. 4 is a flowchart showing an image pickup display processing procedure in the first embodiment.

FIG. 5 is a diagram showing a configuration of a vehicle peripheral display device according to a second embodiment of the present invention.

FIG. 6 is a diagram showing a configuration of a vehicle peripheral display device according to a third embodiment of the present invention.

FIG. 7 is a diagram showing a configuration of a vehicle peripheral display device according to a fourth embodiment of the present invention.

FIG. 8 is a diagram showing an appearance of a vehicle equipped with a conventional vehicle peripheral display device.

FIG. 9 is a diagram showing a configuration of a conventional vehicle peripheral display device.

[Explanation of symbols]

1 irradiation part 2 Light intensity modulator 3 Imaging unit 4 Image recording section 5 Image difference section 6, 6A, 6B brightness correction unit 7 Display 8 Output image judgment unit 9 Multiplier 10 Integrator 11 Average brightness calculation unit 12 Weighting unit 13 Image mixing section

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G03B 17/38 G03B 17/38 B H04N 5/33 H04N 5/33

Claims (11)

[Claims] 1. A light irradiation unit for irradiating light to a display target region, a light amount modulation unit for temporally changing the light amount of the light irradiation unit, and an image of the display target region irradiated by the light irradiation unit. An image difference means for outputting a difference image based on a plurality of images picked up by the image pickup means; and a display means for displaying the difference image outputted by the image difference means, The image difference display unit outputs the difference image based on a plurality of images captured at different timings of the light irradiation unit. 2. The light quantity modulation means controls the light irradiation means so as to periodically repeat lighting and extinguishing, and the imaging means captures the image at each timing of the lighting and extinguishing. Item 1. The image pickup display device according to Item 1. 3. The light quantity modulation means controls the light irradiation means so as to repeat light and dark continuously and periodically, and the image pickup means picks up the image near each peak of the light and dark. 1. The image pickup display device according to 1. 4. A brightness correction unit for correcting the brightness of the difference image output by the image difference unit, wherein the brightness correction unit has a standard brightness a required for the display unit and the image pickup Among the plurality of images captured by the means, let b be the average brightness of the images captured when the light irradiation means has a large amount of light, and c be the average brightness of the images captured when the light irradiation means has a small amount of light. , A / (b-c) is used as the amplification factor to correct the brightness. 5. An output image determination means for comparing the image quality of a raw image captured when the light amount of the light irradiation means is large with an image of the difference output by the image difference means, the output image determination means comprising: The image pickup display device according to claim 1, wherein the image quality is compared based on the sharpness of the edge of the object reflected in the image, and the image with the higher sharpness is output to the display means. 6. An image recording means for recording a raw image taken when the light quantity of the light quantity modulating means is large, an average brightness calculating means for calculating an average brightness of the difference image, and an average brightness calculating means for calculating the average brightness. Based on the average brightness thus calculated, the raw image and the image of the difference are mixed to calculate a mixture ratio when generating one image, and a mixture ratio calculator calculates the mixture ratio. A brightness correction unit that corrects the brightness of each of the two images composed of the raw image and the difference image based on the mixture ratio, and the two images whose brightness has been corrected by the brightness correction unit are added in pixel units. The image display unit according to claim 1, further comprising: an image mixing unit that mixes the mixed images, and the display unit displays the mixed image. 7. A light irradiating unit for irradiating a display target region with light, a light amount modulating unit for outputting modulation information for temporally changing the light amount of the light irradiating unit, and a display target irradiated by the light irradiating unit. Image pickup means for picking up an image of the region, multiplication means for multiplying the modulation information output from the light amount modulation means and the luminance information of the picked-up image at the timing of the image pickup by the image pickup means, and the multiplying means An integrating means for integrating the result of the above over a predetermined time; a brightness correcting means for correcting the brightness of the image obtained as a result of the integration by the integrating means; and an image whose brightness is corrected by the brightness correcting means. An image pickup display device comprising a display means. 8. The display device according to claim 1, wherein the display device is mounted on a vehicle, and a condition around the vehicle is set as the display target area.
The image pickup display device according to any one of 1. 9. A light irradiation step of irradiating the display target area with light while changing the light amount, an imaging step of capturing a plurality of images of the display target area irradiated in the light irradiation step, and a plurality of the captured plurality of images. Based on the image, an image difference step of outputting a difference image, and a display step of displaying the difference image output in the image difference step, in the image difference step, the light amount in the light irradiation step An image pickup display method for outputting a difference image based on a plurality of images picked up at different timings. 10. A light irradiation step of outputting modulation information for temporally changing the amount of light when irradiating the display target area with light, and capturing an image of the display target area irradiated with light in the light irradiation step. An image capturing step, a multiplication step of multiplying the modulation information output at the image capturing timing in the image capturing step and the brightness information of the captured image, and a multiplication result obtained in the multiplying step over a predetermined time. Imaging display having an integration step of integrating, a brightness correction step of correcting the brightness of the image obtained as a result of the integration obtained in the integration step, and a display step of displaying the image corrected in the brightness correction step. Method. 11. A program for causing a computer to execute the image pickup display method according to claim 9.