JP2007036831A - Image display system, image display method and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image display system, an image display method and an imaging apparatus, in which a composite image of a visible image and an infrared image can be clearly displayed without attenuating incident light quantity by switching the incidence of incident light to a visible light imaging device or an infrared light imaging device. <P>SOLUTION: In the image display system for displaying a composite image composed of a visible light image and an infrared light image picked up by an imaging apparatus for picking up an image of the outside of a vehicle; the imaging apparatus is provided with a visible light imaging device for picking up an image of visible light, an infrared light imaging device for picking up an image of infrared light and a switching means for time-dividedly selecting and guiding incident light to the visible light imaging device or the infrared light imaging device to clearly display a composite image of a visible image and an infrared image by setting an interval for switching incident light shorter than time allowed to be viewed by human eyes. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image display system that displays and outputs composite image data obtained by combining visible light data and infrared light data acquired by an imaging device mounted on a vehicle, and an imaging device that constitutes the image display system.

  A system has been proposed in which an imaging device capable of imaging the outside is mounted on a vehicle, and an image captured by the imaging device is displayed on a display device to perform forward monitoring or follow-up vehicle tracking. Image data acquired by an imaging device for visible light using a CCD as an imaging element can display an image that can be easily identified by the driver. However, it is difficult for image data acquired by an imaging device for visible light when traveling at night to provide an image that can identify information other than the irradiation range of the headlight.

  On the other hand, there is also a system that uses an infrared video camera equipped with a bolometer or a pyroelectric image sensor in order to image external information during night driving. However, such a conventional video camera system has a problem that the contrast of the displayed image is low and it is difficult for the driver to identify the presence of an obstacle.

In order to solve such a problem, for example, Non-Patent Document 1 discloses a composite image in which a video camera for visible light and a video camera for infrared light are juxtaposed, and image data acquired by each video camera is synthesized. A display method is disclosed, and a driver can easily identify an obstacle such as a pedestrian from the displayed composite image (see Non-Patent Document 1).
Thomas Widener, et al., “Fusion Algorithm of Far Infrared and Night Visible Images in Automotive Applications”, Baembe Company, 2003 (Thomas Weidner, etc. “Algorithm for Fusion of Far Infrared and Visual Night-Time Images with Scope to Automotive Application ”, BMW, 2003)

  However, in the conventional method described above, it is necessary to place a video camera for visible light and a video camera for infrared light on the vehicle, and an image by visible light (visible image) and an image by infrared light (infrared) Misalignment with the image). Therefore, when a composite image is generated, there is a problem that it is difficult to superimpose an image captured by a visible light video camera and an image captured by an infrared light video camera.

  In order to solve such a problem, incident light is passed through an optical element such as a prism, and is separated into visible light and infrared light. The separated visible light is transferred to an image sensor for visible light, and infrared light is converted into red light. An imaging device that can be guided to an external light imaging device has been developed. In such an imaging apparatus, a visible image and an infrared image can be acquired based on a single incident light, and positional deviation between the visible image and the infrared image can be reduced.

  However, in the imaging apparatus described above, since the incident light is separated into visible light and infrared light by passing through the inside of an optical element such as a prism, particularly far infrared light is attenuated by the passage of the prism. However, there is a problem that it is difficult to display an infrared image clearly.

  The present invention has been made in view of such circumstances, and attenuates the amount of incident light by switching whether incident light is incident on an image sensor for visible light or incident on an image sensor for infrared light. It is another object of the present invention to provide an image display system, an image display method, and an imaging apparatus that can clearly display a composite image of a visible image and an infrared image.

  In addition, the present invention provides an image display system and an image display that can clearly display a composite image of a visible image and an infrared image by making the interval of switching incident light smaller than the time that a human can visually recognize. It is an object to provide a method and an imaging apparatus.

  In order to achieve the above object, an image display system according to a first aspect of the present invention is an image display device that images the outside of a vehicle, and a display that displays a composite image obtained by combining a visible light image and an infrared light image captured by the image capture device. In the image display system, the imaging device includes a visible light imaging element that captures visible light, an infrared light imaging element that captures infrared light, and incident light that is incident on the visible light imaging element. Or a switching means for switching to the infrared imaging device so as to be selected and guided in a time division manner.

  The image display system according to the second invention is characterized in that, in the first invention, the switching means switches at intervals of 100 msec or less.

  The image display system according to a third aspect of the present invention is the image display system according to the first or second aspect, wherein the switching means is a plate-like reflecting mirror, swings about an end portion as a rotation axis, and the optical path of incident light is the visible light. It is characterized by switching to the image sensor for light or the image sensor for infrared light.

  The image display system according to a fourth aspect of the present invention is the image display system according to the first or second aspect, wherein the switching means is a rotator having one or a plurality of reflectors whose surfaces are capable of reflecting light, the rotation axis being The optical path rotates to the center, and the optical path is switched to the visible light image sensor or the infrared light image sensor.

  An image display method according to a fifth aspect of the present invention is an image display method for displaying a composite image obtained by synthesizing a visible light image and an infrared light image captured by an imaging device that captures the outside of a vehicle. And switching to a visible light image sensor that picks up images or an infrared light image sensor that picks up infrared light so as to be selected and guided in a time-sharing manner.

  An image pickup apparatus according to a sixth aspect of the invention is an image pickup apparatus that picks up visible light and infrared light data by picking up the outside of the vehicle, picks up visible light image pickup elements, and picks up infrared light. An infrared light imaging device, and switching means for switching the incident light to the visible light imaging device or the infrared light imaging device so as to be selected and guided in a time-sharing manner. To do.

  In the first invention, the fifth invention, and the sixth invention, the imaging apparatus includes a visible light imaging device that captures visible light, an infrared light imaging device that captures infrared light, and incident light for visible light. And switching means for switching to the image sensor or the infrared image sensor so as to be selected and guided in a time division manner. Thereby, the incident light is guided to either the visible light image sensor or the infrared light image sensor, and outputs a visible image or an infrared image at predetermined time intervals, respectively. Therefore, for example, by switching to a visible image or an infrared image at a short time interval that cannot be visually recognized by humans, the visible image and the infrared image are recognized as afterimages, and a composite image is generated by the image processing device. Without this, the display image can be recognized as a composite image of both. Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

  In the second invention, the switching means switches so as to guide the incident light to the visible light image sensor or the infrared light image sensor at a time interval of 100 msec or less. As a result, by switching to a visible image or an infrared image at a short time interval that cannot be visually recognized by humans, the visible image and the infrared image are recognized as afterimages, and a composite image is generated by the image processing device. Without this, the display image can be recognized as a composite image of both.

  In the third invention, the switching means is a plate-like reflecting mirror, swings about the end portion as the rotation axis, and switches the optical path of the incident light to the visible light image sensor or the infrared light image sensor. Thus, by switching the reflection direction of the reflecting mirror to the direction of the visible light image sensor or the infrared light image sensor at a predetermined time interval, for example, a visible image or infrared light can be displayed at a short time interval that cannot be visually recognized by humans. By switching to the image and outputting it, it becomes possible to recognize the visible image and the infrared image as an afterimage, and to recognize the display image as a composite image of both without generating a composite image by the image processing device. . Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

  In the fourth invention, the switching means is a rotating body having one or a plurality of reflectors whose surfaces are capable of reflecting light, rotates around the rotation axis, and changes the optical path to an imaging element for visible light or infrared light. Switch to the image sensor. Thus, by rotating the rotating body and reflecting incident light at a predetermined time interval in the direction of the visible light image sensor or the infrared light image sensor, for example, a visible image at a short time interval that cannot be seen by humans. Alternatively, by switching to an infrared image and outputting it, the visible image and the infrared image are recognized as afterimages, and the display image can be recognized as a composite image of both without generating a composite image by the image processing device. It becomes possible. Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

  According to the first invention, the fifth invention, and the sixth invention, the incident light is guided to either the visible light image sensor or the infrared light image sensor, and the visible image or the infrared image is respectively transmitted at a predetermined time interval. Output. Therefore, for example, by switching to a visible image or an infrared image at a short time interval that cannot be visually recognized by humans, the visible image and the infrared image are recognized as afterimages, and a composite image is generated by the image processing device. Without this, the display image can be recognized as a composite image of both. Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

  According to the second invention, the visible image and the infrared image are mutually recognized as an afterimage by switching to a visible image or an infrared image at a short time interval that cannot be visually recognized by humans, and the image processing device It is possible to recognize a display image as a composite image of both without generating a composite image.

  According to the third aspect of the invention, by switching the reflection direction of the reflection mirror to the direction of the visible light image sensor or the infrared light image sensor at a predetermined time interval, for example, it is visible at a short time interval that cannot be visually recognized by humans. By switching to an image or infrared image and outputting it, the visible image and the infrared image are recognized as afterimages, and the display image is recognized as a composite image of both without generating a composite image by the image processing device. Is possible. Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

  According to the fourth aspect of the invention, the incident light is reflected in the direction of the visible light image sensor or the infrared light image sensor at a predetermined time interval by the rotation of the rotating body, so that, for example, a short time that cannot be visually recognized by humans. By switching to a visible image or an infrared image at an interval and outputting it, the visible image and the infrared image are recognized as afterimages, and the display image is displayed as a composite image of both without generating a composite image by the image processing device. It becomes possible to recognize. Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

  Hereinafter, the present invention will be described in detail with reference to the drawings illustrating embodiments thereof.

(Embodiment 1)
FIG. 1 is a diagram schematically showing a configuration of an image display system according to Embodiment 1 of the present invention. Reference numerals 1 and 2 denote video cameras (imaging devices) according to the present embodiment for imaging a pedestrian, a person on a bicycle, and the like. The video cameras 1 and 2 are juxtaposed in a substantially horizontal direction at an appropriate length in the front grill of the vehicle. The captured image is transmitted to the image processing apparatus 3 connected via the in-vehicle LAN cable 7 compliant with IEEE1394.

  The image processing device 3 is connected via an in-vehicle LAN cable 7 to the video cameras 1 and 2, a display device 4 having an operation unit, and an alarm device 5 that issues an audible warning by sound, sound effects, and the like. .

  FIG. 2 is a block diagram showing the configuration of the video camera 1 (2) of the image display system according to Embodiment 1 of the present invention. The image pickup unit 11 includes an objective lens 110 that receives light incident from the outside, and a plate-like reflection mirror 111 is provided on the optical axis of the objective lens 110 as a switching unit that switches the optical axis direction of incident light. . The reflection mirror 111 has one end fixed to the rotation axis P, and can swing within a range of a constant rotation angle around the rotation axis P.

  The swinging of the reflection mirror 111 is, for example, a biasing unit that biases the reflection mirror 111 to be in the optical axis direction of the incident light, and the reflection mirror 111 jumps up to a position that blocks the incident light against the biasing unit. This is done with an electromagnetic drive source. That is, when a pulse signal is input to the electromagnetic drive source, the reflection mirror 111 rotates to a position where the incident light is blocked, and otherwise, it is maintained in the optical axis direction of the incident light by the urging means. The swinging means of the reflecting mirror 111 is not limited to this.

  In the optical axis direction of the objective lens 110, an imaging lens 114 having a FIR pass filter attached to the surface is provided, and far-infrared light imaging is performed at a position where infrared light transmitted through the imaging lens 114 forms an image. A pyroelectric imaging element 112 as an element is provided. Note that the far-infrared light image sensor is not limited to the pyroelectric image sensor 112, and may be, for example, a bolometer. The FIR pass filter is a filter that can selectively pass light having a frequency component in the far-infrared region made of, for example, methacrylic resin or polyethylene resin.

  An imaging lens 115 is arranged in a direction in which incident light is reflected when the reflecting mirror 111 rotates and moves at an angle of about 45 degrees from the optical axis direction of the incident light. The CCD 113 is disposed at a position where visible light transmitted through the lens 115 forms an image.

  In order to match the same imaging area with the corresponding imaging surfaces of the pyroelectric imaging device 112 and the CCD 113, the sizes of the imaging surfaces of the pyroelectric imaging device 112 and the CCD 113 and the focal lengths of the imaging lens 114 and the imaging lens 115 are matched. I am letting. Thereby, according to the swing of the reflection mirror 111, the optical axes and the field angles of the infrared light and the visible light with respect to the imaging region can be matched, and the same imaging region via the objective lens 110 is a pyroelectric type. The image is formed on the pixels arranged on the imaging surfaces of the imaging element 112 and the CCD 113.

  When the reflection mirror 111 is rotationally moved in a direction substantially parallel to the optical axis direction of incident light, the incident light is not reflected, and the pyroelectric image sensor is passed through the imaging lens 114 to which the FIR pass filter is attached. 112 is input. The pyroelectric imaging device 112 converts the input infrared light into RGB (R: red, G: green, B: blue) analog signals and outputs the analog signals to the signal processing unit 12.

  Similarly, when the reflecting mirror 111 rotates and moves at an angle of about 45 degrees from the optical axis direction of the incident light and the optical axis direction, the visible light reflected by the reflecting mirror 111 passes to the CCD 113 via the imaging lens 115. Entered. The CCD 113 converts the input visible light into RGB analog signals and outputs them to the signal processing unit 13.

  The signal processing units 12 and 13 convert analog signals input from the pyroelectric imaging device 112 and the CCD 113 into digital signals, respectively, processing for removing various distortions generated in the optical system, and processing for removing low-frequency noise Then, correction processing for correcting the gamma characteristic is performed. Further, the RGB signal is converted into a YUV (Y: luminance, U, V: color difference) signal, and the converted YUV signal is temporarily stored in the image memory 14 as image data.

  The communication interface unit 15 communicates with the image processing device 3 via the in-vehicle LAN cable 7, and the image data temporarily stored in the image memory 14 according to the instruction signal received from the image processing device 3. 3 to send. For example, visible light data and infrared light data temporarily stored in the image memory 14 are transmitted to the image processing device 3 during night driving, and only visible light data temporarily stored in the image memory 14 is transmitted during daytime driving. It transmits to the image processing apparatus 3.

  The control unit 16 is an LSI substrate, for example, and controls operations of the image capturing unit 11, the signal processing units 12 and 13, and the communication interface unit 15. For example, an instruction command (command) received from the image processing device 3 is interpreted, image data temporarily stored in the image memory 14 is read, and output to the image processing device 3 via the communication interface unit 15. The control unit 16 is not limited to an LSI substrate, and may be a CPU, MPU, or the like.

  The control unit 16 controls the rotation angle of the reflection mirror 111 by sending, for example, a pulse signal at regular time intervals to the electromagnetic drive source for rotation of the reflection mirror 111. FIG. 3 is an exemplary diagram of a pulse signal to be transmitted. As shown in FIG. 3, when the output voltage fluctuates to 0 or V0 at every constant time t0 and the output voltage is 0, the electromagnetic drive source does not operate and the reflection mirror 111 does not rotate. When the output voltage is V0, the electromagnetic drive source is activated and the repulsive force is controlled so that the rotation angle of the reflection mirror 111 is approximately 45 degrees against the urging means. Accordingly, the same imaging region can be switched to be guided to the pyroelectric imaging device 112 or the CCD 113 at regular time intervals, and the infrared light data or the visible light data is respectively transmitted via the communication interface unit 15 to the image processing apparatus. Output to 3.

  The time interval for switching the rotation angle of the reflection mirror 111 is set to be shorter than the time that humans can visually recognize. For example, by setting the time interval to 100 msec or less, even when visible light data is displayed on the display device 4 and then infrared light data is displayed, the visible light data remains as an afterimage, and the image Without generating a composite image in the processing device 3, a human recognizes that composite image data of visible light data and infrared light data is displayed.

  FIG. 4 is a block diagram showing a configuration of the image processing apparatus 3 according to Embodiment 1 of the present invention. The communication interface unit 31 transmits commands to the video cameras 1 and 2 and receives image data from the video cameras 1 and 2. The communication interface unit 31 stores the image data received from the video cameras 1 and 2 in the image memory 32 in units of frames.

  The communication interface unit 31 transmits image data to the display device 4 such as a liquid crystal display, and transmits an output signal such as a synthesized sound to the alarm device 5 such as a buzzer or a speaker. The image memory 32 is an SRAM, a flash memory, or the like, and stores image data received from the video cameras 1 and 2 via the communication interface unit 31.

  The LSI 33 is a substrate that performs image processing, reads visible light data and infrared light data stored in the image memory 32 in units of frames, and reads the read visible light data and infrared light data via the communication interface unit 31. To the display device 4. Further, the LSI 33 has a built-in RAM 331 for storing data generated during the arithmetic processing.

  Visible light data and infrared light data are displayed on the display device 4. The display device 4 uses the afterimage phenomenon of human eyes without causing the image processing device 3 to generate a composite image of visible light data and infrared light data by swinging the reflecting mirror 111 at the coincidence time interval. Thus, the composite image is displayed.

  FIG. 5 is a diagram illustrating a display example of an image captured by the video camera 1 (2) during night driving. FIG. 5A is an exemplary diagram in the case where visible light data obtained by imaging by the CCD 113 is displayed on the display device 4, and FIG. 5B is obtained by imaging by the pyroelectric imaging device 112. It is an illustration figure at the time of displaying the obtained infrared light data. FIG. 5C is an exemplary diagram showing a state recognized as a composite image in which both are combined by the afterimage phenomenon.

  In FIG. 5A, when the video camera 1 (2) receives reflected light from the front road surface, road shoulder or the like irradiated with the headlight of the vehicle, it is visible from the imaging area such as the road shoulder or the lane change lane. The optical data has a high luminance value, and the visible light data from the imaging area where the headlight is not irradiated has a low luminance value. Accordingly, road shoulders, lane change lanes, and the like can be displayed to such an extent that they can be visually confirmed, while other imaging areas in front of the road are hardly displayed.

  In FIG. 5B, when the video camera 1 (2) receives the thermal energy of the object in the imaging region as infrared light, the object such as a road surface, a shoulder, a pedestrian, and surrounding trees in the imaging region in front of the vehicle Since the difference in heat energy from the image is displayed as an image, an object that cannot be recognized by visible light data can be displayed to the extent that it can be individually recognized. For example, when receiving visible light, it is difficult to recognize a pedestrian at night, whereas when receiving infrared light, a pedestrian can be easily recognized by the heat energy emitted from the human body. it can. However, since there is no difference in thermal energy from the same object, an image based on infrared light data has a lower contrast than an image based on visible light data, and there is a region that cannot be visually confirmed.

  In the present embodiment, by swinging the reflection mirror 111 at regular time intervals, the display screen based on visible light data and the display screen based on infrared light data are switched at regular time intervals. Since the switching time interval is shorter than the human visual recognition time, the composite screen as shown in FIG. 5C in which the display screen based on the visible light data and the display screen based on the infrared light data are combined by the afterimage phenomenon. It is recognized as if an image is displayed. Therefore, it is possible to display not only pedestrians that are not irradiated with headlights but also images of surrounding objects, as well as reflected light from objects irradiated with headlights, and can easily recognize the presence of obstacles with high contrast. It is recognized that a composite image that can be displayed is displayed.

  Note that the objective lens 110, the imaging lens 114 for imaging infrared light, and the imaging lens 115 for lacking visible light are not particularly limited in material, and may be a germanium lens, which has a low manufacturing cost. A ZnS lens that can be reduced may be used. Furthermore, the visible light imaging element 113 is not limited to the CCD type, and may be, for example, a CMOS type.

  In the above-described embodiment, the video camera and the image processing apparatus are connected via the in-vehicle LAN cable 7 compliant with IEEE 1394. However, the present invention is not limited to this, and the NTSC is connected to the NTSC. A compliant communication method may be used.

  Further, in the above description, it is assumed that the pyroelectric imaging device 112 and the CCD 113 have the same resolution, but the present invention is not particularly limited thereto, and the resolution of the pyroelectric imaging device 112 and the CCD 113 is respectively It may be different. In addition, the scanning order of the imaging surface (for example, scanning from the upper left to the lower right, or scanning from the lower right to the upper left) may be different between the pyroelectric imaging device 112 and the CCD 113, The scanning method (for example, interlace method and non-interlace method) may be different between the pyroelectric image sensor 112 and the CCD 113.

  When the scanning methods are different, if the frame periods of the pyroelectric imaging device 112 and the CCD 113 are the same, the image data output from the pyroelectric imaging device 112 and the CCD 113 may be output in correspondence with each frame. it can.

  As described above, according to the first embodiment, the incident light is guided to either the visible light image sensor or the infrared light image sensor by changing the angle of the reflection mirror at regular time intervals. A visible image or an infrared image is output at a predetermined time interval. Therefore, by switching to a visible image or an infrared image at a short time interval that cannot be visually recognized by humans, the visible image and the infrared image are recognized as afterimages, and a composite image is generated by the image processing device. Therefore, the display image can be recognized as a composite image of both. Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

(Embodiment 2)
Since the configuration of the image display system according to the second embodiment of the present invention is the same as that of the first embodiment, detailed description is omitted by giving the same reference numerals to components having the same functions. . The second embodiment uses a bolometer instead of a pyroelectric image sensor as the far-infrared light image sensor, and is different in that the incident light guiding path is switched by a reflector provided in the rotating body.

  FIG. 6 is a block diagram showing the configuration of the video camera 1 (2) of the image display system according to Embodiment 2 of the present invention. The image pickup unit 11 includes an objective lens 110 that receives light incident from the outside, and a semicircular reflector 117 is provided on the optical axis of the objective lens 110 as switching means for switching the optical axis direction of incident light. ing. The reflector 117 has a semicircular center point fixed to the rotating shaft of the motor M1, and can rotate at a constant rotational speed around the rotating shaft.

  FIG. 7 is a perspective view of reflector 117 of the image display system according to Embodiment 2 of the present invention. As shown in FIG. 7 (a), the reflector 117 has a semicircular shape, and any one surface is mirror-coated. The center point of the semicircle is fixed to the rotating shaft 120 of the motor M1, and the reflector 117 is rotated by the rotation of the motor M1. Then, the mirror-coated surface is disposed toward the objective lens 110 side. Therefore, when the motor M1 rotates at a constant rotational speed, the incident light travels straight when the reflector 117 is in a position that does not block the incident light, and when the reflector 117 blocks the incident light, a mirror coating is applied. Incident light is reflected in another direction by the surface.

  The reflector 116 is not limited to a semicircular disk shape as shown in FIG. 7A, and is, for example, a plurality of wing shapes as shown in FIG. Any shape may be used as long as the ratio of the center angle to the center of rotation where 117b is present and the center angle not present is 1: 1. In FIG. 7B, two wing-like reflectors 117a and 117b are fixed to the rotating shaft 120, and the center angle θ of the wing-like reflectors 117a and 117b is approximately 90 degrees. Therefore, every time the rotating shaft 120 rotates 90 degrees, the guiding direction of the incident light can be switched.

  In the optical axis direction of the objective lens 110, an imaging lens 114 having a FIR pass filter attached to the surface is provided, and far-infrared light imaging is performed at a position where infrared light transmitted through the imaging lens 114 forms an image. A bolometer 116 as an element is provided. The FIR pass filter is a filter that can selectively pass light having a frequency component in the far-infrared region made of, for example, methacrylic resin or polyethylene resin.

  Further, the imaging lens 115 is disposed in a direction substantially orthogonal to the optical axis direction of the incident light, and the CCD 113 is disposed at a position where the visible light transmitted through the imaging lens 115 forms an image.

  Also in the second embodiment, the sizes of the imaging surfaces of the bolometer 116 and the CCD 113 and the focal lengths of the imaging lens 114 and the imaging lens 115 are matched so that the same imaging area matches the corresponding imaging surfaces of the bolometer 116 and the CCD 113. ing. Thereby, according to the rotation of the reflector 117, the optical axes and the angles of view of the infrared light and the visible light with respect to the imaging region can be matched, and the same imaging region via the objective lens 110 becomes the bolometer 116 and the CCD 113. The image is formed on the pixels arranged on the respective imaging surfaces.

  When the reflector 117 rotates and does not block the incident light, the incident light is input to the bolometer 116 via the imaging lens 114 to which the FIR pass filter is attached. The bolometer 116 converts the input infrared light into RGB (R: red, G: green, B: blue) analog signals and outputs the analog signals to the signal processing unit 16.

  Similarly, when the reflector 117 rotates and reflects incident light, visible light reflected by the reflector 117 is input to the CCD 113 via the imaging lens 115. The CCD 113 converts the input visible light into RGB analog signals and outputs them to the signal processing unit 13.

  The signal processing units 16 and 13 convert analog signals input from the bolometer 116 and the CCD 113 into digital signals, respectively, processing for removing various distortions generated in the optical system, low-frequency noise removal processing, and gamma characteristics. A correction process to correct is performed. Further, the RGB signal is converted into a YUV (Y: luminance, U, V: color difference) signal, and the converted YUV signal is temporarily stored in the image memory 14 as image data.

  The communication interface unit 15 communicates with the image processing device 3 via the in-vehicle LAN cable 7, and the image data temporarily stored in the image memory 14 according to the instruction signal received from the image processing device 3. 3 to send. For example, visible light data and infrared light data temporarily stored in the image memory 14 are transmitted to the image processing device 3 during night driving, and only visible light data temporarily stored in the image memory 14 is transmitted during daytime driving. It transmits to the image processing apparatus 3.

  The control unit 16 is an LSI substrate, for example, and controls operations of the image capturing unit 11, the signal processing units 12 and 13, and the communication interface unit 15. For example, an instruction command (command) received from the image processing device 3 is interpreted, image data temporarily stored in the image memory 14 is read, and output to the image processing device 3 via the communication interface unit 15. The control unit 16 is not limited to an LSI substrate, and may be a CPU, MPU, or the like.

  Based on the instruction command (command) received from the image processing device 3, the control unit 16 sets the rotational angular velocity of the reflector 117. The rotational angular velocity to be set is set so that the time interval at which the output image data is switched to the visible light data or infrared light data by the rotation of the reflector 117 is shorter than the time that humans can visually recognize. For example, when the rotation angular velocity is set so that the time interval for switching to visible light data or infrared light data is 100 msec or less, visible light data is displayed on the display device 4 and then infrared light data is displayed. Even if it exists, visible light data remains as an afterimage, and a human being displays composite image data of visible light data and infrared light data without generating a composite image in the image processing device 3. recognize.

  Note that the objective lens 110, the imaging lens 114 for imaging infrared light, and the imaging lens 115 for lacking visible light are not particularly limited in material, and may be a germanium lens, which has a low manufacturing cost. A ZnS lens that can be reduced may be used. Furthermore, the visible light imaging element 113 is not limited to the CCD type, and may be, for example, a CMOS type.

  In the above-described embodiment, the video camera and the image processing apparatus are connected via the in-vehicle LAN cable 7 compliant with IEEE 1394. However, the present invention is not limited to this, and the NTSC is connected to the NTSC. A compliant communication method may be used.

  Further, in the above description, it is assumed that the resolutions of the bolometer 116 and the CCD 113 are the same, but the present invention is not particularly limited to this, and the resolutions of the bolometer 116 and the CCD 113 may be different. Further, the scanning order of the imaging surface (for example, scanning from the upper left to the lower right direction, or scanning from the lower right to the upper left) may be different between the bolometer 116 and the CCD 113, or the bolometer 116 and the CCD 113. The scanning method (for example, interlace method and non-interlace method) may be different.

  When the scanning methods are different, if the frame periods of the bolometer 116 and the CCD 113 are the same, the image data output from the bolometer 116 and the CCD 113 can be output in correspondence with one frame unit.

  Further, the driving source for rotating the reflector 117 is not limited to the motor M1, and the reflector 117 may be rotated by magnetic field control using a magnet or the like, for example.

  As described above, according to the second embodiment, incident light is reflected in the direction of the visible light image sensor or the infrared light image sensor at a predetermined time interval by the rotation of the rotating body. By switching to a visible image or an infrared image at a short time interval that cannot be performed, the visible image and the infrared image are mutually recognized as afterimages, and without generating a composite image by the image processing device, A display image can be recognized as a composite image. Further, since it does not pass through an optical element such as a prism, the amount of far-infrared light is not particularly attenuated, and a clear composite image can be displayed.

  In the first and second embodiments described above, an example in which the optical axis direction of incident light is switched by approximately 90 degrees has been described. However, the switching angle in the optical axis direction depends on the imaging element for visible light and infrared light. However, the present invention is not limited to this embodiment. Therefore, for example, in order to reduce the outer shape of the video camera, an image sensor for visible light and an image sensor for infrared light may be juxtaposed. In this case, the switching angle in the optical axis direction is 20 degrees. It is about thirty degrees.

It is a figure which shows typically the structure of the image display system which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the video camera of the image display system which concerns on Embodiment 1 of this invention. It is an illustration figure of the pulse signal to send out. 1 is a block diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. It is a figure which shows the example of a display of the image which the video camera image | photographed at the time of night driving | running | working. It is a block diagram which shows the structure of the video camera of the image display system which concerns on Embodiment 2 of this invention. It is a perspective view of the reflector of the image display system which concerns on Embodiment 2 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 2 Video camera 3 Image processing apparatus 4 Display apparatus 11 Image pick-up part 110 Objective lens 111 Reflecting mirror 112 Pyroelectric image sensor 113 CCD
114, 115 Imaging lens 116 Bolometer 117 Reflector 120 Rotating axis

Claims (6)

In an image display system comprising: an imaging device that images the outside of a vehicle; and a display device that displays a composite image obtained by combining a visible light image and an infrared light image captured by the imaging device.
The imaging device
An image sensor for visible light that images visible light;
An infrared image sensor for imaging infrared light; and
An image display system comprising: a switching unit that switches the incident light to the visible light image sensor or the infrared light image sensor so as to be selected and guided in a time division manner.   2. The image display system according to claim 1, wherein the switching means switches at intervals of 100 msec or less.   The switching means is a plate-like reflection mirror, swings about an end portion as a rotation axis, and switches the optical path of incident light to the visible light imaging element or the infrared light imaging element. The image display system according to claim 1 or 2, characterized in that   The switching means is a rotating body having one or a plurality of reflectors whose surfaces can reflect light, and rotates about a rotation axis, and the optical path is the imaging device for visible light or the imaging for infrared light. 3. The image display system according to claim 1, wherein the image display system is switched to an element. In an image display method for displaying a composite image obtained by synthesizing a visible light image and an infrared light image captured by an imaging device that captures the outside of a vehicle,
An image display method characterized by switching incident light to a visible light imaging device that captures visible light or an infrared light imaging device that captures infrared light so as to be selected and guided in a time division manner. In an imaging device that captures the outside of a vehicle and acquires visible light data and infrared light data,
An image sensor for visible light that images visible light;
An infrared image sensor for imaging infrared light; and
An image pickup apparatus comprising: a switching unit that switches incident light to the visible light image sensor or the infrared light image sensor so as to select and guide the incident light in a time division manner.

Images