JP2006109171A - Display device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform highlighting of a three-dimensional object by simple structure. <P>SOLUTION: This display device for vehicle is characterized in that infrared lamps 2a, 2b emit light from different directions on both left and right sides by shifting timing, a CCD camera 3 combines video data picked up when the infrared lamp 2a is lit with video data picked up when the infrared lamp 2b is lit to generate combined video data and displays and outputs the generated combined video data. Thus, since a shadow is display on both sides of the object in the combined video data, the object around a vehicle is highlighted by boundary display without requiring special image processing and a special hardware configuration. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a vehicular display device that captures and displays a target object around a vehicle such as a pedestrian or an obstacle at night or the like.

  2. Description of the Related Art Conventionally, a vehicular display device that captures and displays the front of a vehicle with two headlamps turned on simultaneously is known.

  In this conventional vehicle display device, by performing image processing on the captured image, a contour of a three-dimensional target body such as a pedestrian or an obstacle existing in front of the vehicle is extracted, and a pedestrian or the like is extracted. I'm trying to highlight it.

  However, since the image processing is based on simple video, there is a problem that the processing becomes enormous and the processing accuracy is not good.

  In addition, when an infrared lamp is used as a lighting device, for example, a pedestrian emerges in white with respect to a black background as shown in FIG. 16 (a), but the background is shown in FIG. 16 (b). If a bright object enters, the bright part of the background and the pedestrian are assimilated, which may make it difficult to recognize the pedestrian.

  On the other hand, there are some which display a target object in a three-dimensional manner by imaging the front of the vehicle using a plurality of imaging devices.

  However, since a plurality of imaging devices are required, the cost of the vehicle display device increases, and a problem arises in the installation location in the vehicle.

JP 2004-94825 A

  The problem to be solved is that the three-dimensional target object cannot be highlighted with a simple structure.

  In the present invention, in order to highlight a three-dimensional target with a simple structure, a plurality of lighting devices that irradiate light from different directions to a target outside the vehicle are turned on at different timings, and each captured image is displayed. The most important feature is that the composite video is displayed on the display device.

  The vehicle display device of the present invention is configured to turn on a plurality of illumination devices that irradiate light from different directions onto a target body outside the vehicle, and to display and output a composite image of each captured image on the display device. Boundaries can be displayed on the target object in the composite image by shadows formed on the side edges of the three-dimensional target object when each lighting device is turned on, and the vehicle does not require special image processing or hardware configuration. The surrounding three-dimensional target body can be highlighted.

  When the imaging apparatus adds a boundary display to the target body of the composite video based on the difference video based on each video, the shadow formed on the side edge of the target body is reliably captured, and the boundary display is displayed on the target body. It can be done reliably.

  When the imaging apparatus emphasizes the boundary display, the target body can be surely highlighted.

  The imaging apparatus performs a first binarization process on the difference video using a first threshold value and performs a second binarization process on a multiplied video obtained by multiplying the synthesized video by a second threshold value, When the boundary display is added to the target body of the composite video based on the selection process of the inverted video after the binarization processing and the composite video, the shadow portion data is surely extracted and the boundary of the target body is extracted. The display can be performed more reliably.

  When the lighting devices are mounted on both the left and right sides of the vehicle, and the switching device shifts the timing in synchronization with a field pulse input from the imaging device, the left and right lighting devices are turned on for each field of the imaging device. By doing so, it is possible to easily display the boundary of the target body.

  When the illumination device emits near-infrared light, the boundary display of the target body can be reliably performed at night or the like.

  The objective of highlighting a three-dimensional target with a simple structure has been realized by lighting a plurality of lighting devices that emit light from different directions at different timings.

[Vehicle display device]
1 to 6 relate to the vehicle display device of the first embodiment, FIG. 1 is a block diagram showing the configuration of the vehicle display device, and FIG. 2 is an arrangement position of the infrared lamp and the CCD camera shown in FIG. FIG. 3 is a block diagram showing the internal configuration of the CCD camera shown in FIG. 1, FIG. 4 is a circuit diagram showing the internal configuration of the switching circuit shown in FIG. 1, and FIG. FIG. 6 is a flow chart for explaining the operation of the switching circuit shown in FIG. 1, and FIG. 6 is a timing chart for explaining the operation of the switching circuit shown in FIG.

  The vehicle display device 1 in FIG. 1 is mounted on a vehicle and includes infrared lamps 2a and 2b, a CCD (Charge Coupled Device) camera 3, a switching circuit 4, and a display device 5 as main components.

  As shown in FIG. 2, the infrared lamps 2a and 2b are disposed on the left and right sides of the front part of the vehicle. The infrared lamps 2a and 2b irradiate near-infrared light ahead of the vehicle according to the control of the switching circuit 4 in response to the lighting switch SW (see FIG. 4) being turned on. The optical axes of the infrared lamps 2a and 2b are set so as to cross each other in front of the vehicle, and irradiate the target body, for example, a pedestrian, from both left and right sides in different directions.

  Therefore, it has a configuration provided with a plurality of illumination devices (infrared lamps 2a, 2b) that irradiate the target body outside the vehicle with light from different directions.

  An infrared LED (Light Emission Diode) may be used instead of the infrared lamp. In addition, it is desirable that the infrared lamps 2 a and 2 b be arranged symmetrically with respect to the imaging direction of the CCD camera 3.

  The CCD camera 3 is disposed on the rear surface of the rearview mirror in the vehicle (see FIG. 2), and images the front of the vehicle in the vehicle exterior direction. Therefore, the image pickup apparatus (CCD camera 3) for picking up an image in the vehicle exterior direction is provided.

  As shown in FIG. 3, the CCD camera 3 includes a CCD 11, a DSP (Digital Signal Processor) 12, an input image memory 13, a calculation memory 14, a superimpose memory 15, a video switching unit 16, and a CPU (Central Processing Unit). ) 17. The CCD camera 3 outputs field pulses to the switching circuit 4 so that the infrared lamps 2a and 2b emit pulses alternately. The CCD camera 3 captures an image of the front of the vehicle with the infrared lamps 2a and 2b alternately emitting pulses, and executes a boundary display process (to be described later) on the captured image. The target object around the vehicle is highlighted three-dimensionally. The function of each component of the CCD camera 3 will be described later.

  The switching circuit 4 constitutes a switching device, and as shown in FIG. 4, an input terminal IN to which a field pulse from the CCD camera 3 is input, a transistor T1 connected to an infrared lamp 2a on the left side in front of the vehicle, And a transistor T2 connected to the infrared lamp 2b on the right front side of the vehicle.

  5 and 6, the switching circuit 4 is in a high state (EVEN field) when the field pulse input from the input terminal IN (FIG. 4) is set (YES in step S1 in FIG. 5). The transistors T1 and T2 (FIG. 4) are switched between the off state and the on state. By this switching, the infrared lamp 2a (left side illumination) is switched off and the infrared lamp 2b (right side illumination) is switched on (step S2 in FIG. 5).

  When the field pulse input from the input terminal IN is in the low state (ODD field) (step S1 in FIG. 5 is NO), the transistor T1 and the transistor T2 are switched to the on state and the off state. By this switching, the infrared lamp 2a is switched on and the infrared lamp 2b is switched off (step S3 in FIG. 5).

  Accordingly, the lighting device (infrared lamps 2a and 2b) is provided with a switching device (switching circuit 4) that turns on the lighting at different timings. The switching device (switching circuit 4) is a field input from the imaging device (CCD camera 3). The timing is shifted by synchronization with the pulse.

  The display device 5 is composed of a vehicle head-up display or the like, and displays video on the windshield. The display device 5 displays and outputs, for example, an image in front of the vehicle, which is around the vehicle image-processed by the CCD camera 3 in accordance with an NTSC (National Television Standards Committee) signal input from the CCD camera 3. The display device 5 can also use other display configurations such as a liquid crystal display device.

  The vehicle display device 1 having such a configuration performs boundary display on the target body around the vehicle without executing special image processing or hardware configuration by executing the boundary display processing described below. .

  Hereinafter, with reference to the flowchart shown in FIG. 7, operation | movement of the display apparatus 1 for vehicles at the time of performing a boundary display process is demonstrated.

In addition, FIGS. 8-15 is referred as needed. FIG. 8 is a diagram illustrating an example of image data captured when the infrared lamps on the left front side of the vehicle and the right front side of the vehicle are lit. FIG. FIG. 10 is a diagram illustrating an example of video data obtained by processing. FIG. 10 is a diagram illustrating an example of video data obtained by performing binarization processing on video data obtained by performing difference processing. 11 is a diagram showing an example of video data obtained by performing expansion processing on the video data shown in FIG. 9, and FIG. 12 shows video data captured when the infrared lamps on the front left side and the front right side of the vehicle are turned on. FIG. 13 is a diagram showing an example of video data obtained by combining, and FIG. 13 is a video obtained by multiplying the video data shown in FIG. 10 and the video data shown in FIG. FIG. 14 is a diagram illustrating an example of superimpose data. FIG. 15 is a diagram illustrating an example of video data obtained by selecting the video data and superimpose data in FIG. FIG.
[Boundary display processing]
The flowchart shown in FIG. 7 starts when the power source of the vehicle display device is turned on and the CCD camera 3 alternately pulses the infrared lamps 2a and 2b, and the boundary display processing is performed in step S11. move on.

  In the process of step S <b> 11, the CCD 11 captures an image in front of the vehicle and outputs data of the captured image to the DSP 12. Thereby, the process of step S11 is completed, the process of step S11 is completed, and the boundary display process proceeds from the process of step S11 to the process of step S12.

  In step S12, the DSP 12 performs signal processing such as analog / digital conversion processing on the video data input from the CCD 11. In this embodiment, each pixel of the video data has a density value of 256 (0 to 255) gradations, a pixel whose density value is 0 is displayed in black, and a pixel whose density value is 255 is displayed in white. Is done. Thereby, the process of step S12 is completed, and the boundary display process proceeds from the process of step S12 to the process of step S13.

  In the process of step S <b> 13, the DSP 12 records the video data subjected to the signal processing in the input image memory 13. Thereby, the process of step S13 is completed, and the boundary display process proceeds from the process of step S13 to the process of step S14.

  In the process of step S <b> 14, the CPU 17 determines whether one frame of video data has been recorded in the input image memory 13. If the result of determination is that one frame of video data has not been recorded, the CPU 17 returns the boundary display process to the process of step S11. On the other hand, if video data for one frame is recorded, the CPU 17 advances the boundary display process to the process of step S15.

  In the process of step S15, the CPU 17 calculates a difference value between the density value of each pixel of the video data captured when the infrared lamp 2a is lit and the density value of each pixel of the video data captured when the infrared lamp 2b is lit. The absolute value of the calculated difference value is calculated. Then, the CPU 17 records the video data representing the calculated absolute value in the calculation memory 14 as a difference video.

  Specifically, when the image data captured when the infrared lamp 2a is turned on and when the infrared lamp 2b is turned on is shown in FIGS. 8A and 8B, the CPU 17 determines the density of each pixel constituting the image data. By calculating the difference value of the values and calculating the absolute value of the calculated difference value, video data as shown in FIG. 9 is obtained as the difference video. Thereby, the process of step S15 is completed, and the boundary display process proceeds from the process of step S15 to the process of step S16.

  In the process of step S16, the CPU 17 performs a first binarization process on the video data recorded in the calculation memory 14, for example, using the density value 1 as a first threshold value. Pixels having a density value of less than 1 are converted to a density value of 0 (black) and those having a density value of 1 or more are converted to a density value of 255 (white), and binarized video data is recorded in the calculation memory.

  Specifically, when the video data shown in FIG. 9 is recorded in the arithmetic memory 14, the CPU 17 obtains video data as shown in FIG. 10 by the first binarization process.

  In this embodiment, the CPU 17 performs the first binarization process using the density value 1 as a threshold value, but other density values may be used as the threshold value. Thereby, the process of step S16 is completed, and the boundary display process proceeds from the process of step S16 to the process of step S17.

  In the process of step S17, the CPU 17 performs the expansion process on the binarized video data by replacing the black pixel adjacent to the pixel having the density value of 255 (white) with the white pixel, and the expansion process is performed. The video data is recorded in the calculation memory 14. According to such processing, the video data shown in FIG. 10 is converted into video data from which noise has been removed as shown in FIG. The black portion of the video data in FIG. 11 is slightly smaller than that in FIG. Thereby, the process of step S17 is completed, and the boundary display process proceeds from the process of step S17 to the process of step S18.

  In the process of step S18, the CPU 17 generates a composite video by combining the video data captured when the infrared lamp 2a is lit and the video data captured when the infrared lamp 2b is lit, and the composite video data and the expansion process are performed. Video data multiplied by the video data is generated as a multiplied video.

  Specifically, when the image data captured when the infrared lamp 2a is turned on and when the infrared lamp 2b is turned on is as shown in FIGS. 8A and 8B, the CPU 17 performs the processing shown in FIGS. 12 is generated by combining the video data shown in FIG. 12, and the video data shown in FIG. 13 is generated by multiplying the composite video data by the video data shown in FIG.

  According to such processing, a white pixel is converted into a black pixel in one of the synthesized video data or the video data after the expansion process, and the contour line L of the pedestrian as the target body included in the video data is changed. Displayed in white. Then, the CPU 17 records the video data obtained by the multiplication in the calculation memory 14. Thereby, the process of step S18 is completed, and the boundary display process proceeds from the process of step S18 to the process of step S19.

  In the process of step S19, the CPU 17 performs a second binarization process on the video data generated by the process of step S18 using the density value 230 as a second threshold value. Pixels having a density value of less than 230 are converted to black, and those having a density value of 230 or more are converted to white, whereby the contour line L of the target body contained in the video data is extracted, and the video data of the extracted contour line L is extracted. Is recorded in the superimpose memory 15 as superimpose data.

  Specifically, when the video data as shown in FIG. 13 is generated by the process of step S18, the CPU 17 performs the second binarization process on the video data as shown in FIG. Video data of the contour line L is generated as superimpose data.

  In this embodiment, the CPU 17 performs the second binarization process using the density value 230 as the second threshold value, but other density values may be used as the threshold value. Thereby, the process of step S19 is completed, and the boundary display process proceeds from the process of step S19 to the process of step S20.

  In the process of step S <b> 20, the CPU 17 determines whether or not superimpose data is recorded in the superimpose memory 15. If the result of the determination is that superimpose data is not recorded, the video switching unit 16 outputs the composite video data to the DSP 12 as the process of step S21, and then advances the boundary display process to the process of step S23. On the other hand, when superimpose data is recorded, the video switching unit 16 outputs the composite video data and the superimpose data to the DSP 12 as the process of step S22, and then advances the boundary display process to the process of step S23. .

  In step S23, the DSP 12 digital / analog converts the video data input from the video switching unit 16, and then outputs the NTSC signal to the display device 5 to display the video data on the display device 5.

  Specifically, when the composite video data is input, the DSP 12 causes the display device 5 to display and output the composite video data as shown in FIG. On the other hand, when the composite video data and the superimpose data are input, the DSP 12 generates video data obtained by inverting the superimpose data, and the density value of the pixels constituting the inverted video data and the composite video data is the same. A selection process for extracting a low pixel is performed. With this selection processing, a black contour line L as shown in FIG. 15 is displayed as a boundary on the pedestrian as the target body, and the video data highlighted by this boundary display is displayed on the display device 5.

  At this time, the DSP 13 may give a color to the contour line L and highlight the contour line. As the color type, it is preferable to select a conspicuous color such as red. Thereby, the process of step S23 is completed, and the boundary display process proceeds from the process of step S23 to the process of step S24.

  In the process of step S24, the DSP 12 determines whether or not the process for video data for one frame has been completed. If the result of determination is that processing for video data for one frame has not been completed, the DSP 12 returns the boundary display processing to the processing in step S15. On the other hand, when the process for the video data for one frame is completed, the DSP 12 returns the boundary display process to the process of step S11.

  As described above, according to the vehicle display device 1 of the first embodiment, the CCD camera 3 combines and synthesizes video data captured when the infrared lamp 2a is lit and video data captured when the infrared lamp 2b is lit. Video data is generated, and the generated composite video data is displayed and output. According to such a configuration, as shown in FIG. 12, shadows are displayed on both side edges of the pedestrian in the composite video data, so that special image processing and hardware configuration are required. The pedestrian in front of the vehicle can be highlighted by the boundary display. Further, when a shadow is displayed on the pedestrian, a boundary line is generated between the background and the pedestrian, and the pedestrian can be accurately recognized from the video data.

  The imaging apparatus can highlight the boundary display by the shadow, and can reliably highlight the target body. That is, since the CCD camera 3 selects and processes the superimpose data representing the contour line L of the target body and the composite video data, it is easy to determine a three-dimensional object from the video data as shown in FIG. It becomes.

  Since the infrared lamps 2a and 2b irradiate near infrared light, the boundary display of a pedestrian in front of the vehicle can be reliably performed even at night.

  Note that it is possible to provide three or more lighting devices as long as they emit light from different directions. The lighting timing can also be performed by a signal other than the field pulse.

1 is a block diagram illustrating a configuration of a vehicle display device according to an embodiment of the present invention (Example 1). FIG. FIG. 2 is a front view of an automobile for explaining the arrangement positions of the infrared lamp and the CCD camera shown in FIG. 1 (Example 1). FIG. 2 is a block diagram showing an internal configuration of the CCD camera shown in FIG. 1 (Example 1). FIG. 2 is a circuit diagram showing an internal configuration of the switching circuit shown in FIG. 1 (Example 1). FIG. 3 is a flowchart for explaining the operation of the switching circuit shown in FIG. 1 (Example 1). FIG. 3 is a timing chart for explaining the operation of the switching circuit shown in FIG. 1 (Example 1). It is a flowchart figure which shows the flow of the boundary display process used as one Embodiment of this invention (Example 1). (Example 1) which is a figure which shows an example of the video data imaged at the time of the infrared lamp lighting of the vehicle front left side and the vehicle front right side. (Example 1) which is a figure which shows an example of the video data obtained by carrying out the difference process of the video data imaged at the time of the infrared lamp lighting of the vehicle front left side and the vehicle front right side. (Example 1) which is a figure which shows an example of the video data obtained by performing the binarization process with respect to the video data obtained by performing a difference process. FIG. 10 is a diagram illustrating an example of video data obtained by performing expansion processing on the video data illustrated in FIG. 9 (Example 1). (Example 1) which is a figure which shows an example of the video data obtained by synthesize | combining the video data imaged at the time of the infrared lamp lighting of the vehicle front left side and the vehicle front right side. FIG. 12 is a diagram illustrating an example of video data obtained by multiplying the video data illustrated in FIG. 10 and the video data illustrated in FIG. 11 (Example 1). (Example 1) which is a figure which shows an example of superimpose data. FIG. 13 is a diagram illustrating an example of video data obtained by a selection process between video data and superimpose data in FIG. 12 (Example 1). (A) is a figure which shows the video data of pedestrian recognition, (b) is a figure of the video data which binarized (a) (conventional example).

Explanation of symbols

1: Vehicle display devices 2a, 2b: Infrared lamp (lighting device)
3: CCD (Charge Coupled Device) camera (imaging device)
4: Switching circuit 5: Display device

Claims (6)

An imaging device for imaging the direction outside the vehicle;
A plurality of lighting devices for irradiating the target body outside the vehicle with light from different directions;
A switching device that turns on each lighting device at different timings; and
A display device for displaying and outputting the video imaged by the imaging device,
The imaging device causes the display device to display and output a composite video of each video imaged when the lighting devices are turned on. The vehicle display device according to claim 1,
The imaging apparatus adds a boundary display to the target body of the composite video based on the difference video based on each video. The vehicle display device according to claim 2,
The imaging apparatus emphasizes the boundary display. A vehicle display device. The vehicle display device according to claim 2,
The imaging apparatus performs a first binarization process on the difference image using a first threshold value, performs a second binarization process on a multiplication image obtained by multiplying the synthesized image by a second threshold value, A boundary display is added to a target body of the composite video based on a selection process of the inverted video after the binarization processing and the composite video. The vehicle display device according to any one of claims 1 to 4,
The lighting device is attached to the left and right sides of the vehicle,
The display device for a vehicle, wherein the switching device shifts the timing by synchronization with a field pulse input from the imaging device. The vehicle display device according to any one of claims 1 to 5,
The illumination device irradiates near infrared light. A vehicle display device.

Images