JP5105306B2 - Vehicle display device - Google Patents

Description

  The present invention relates to a vehicle display device that displays an image around a vehicle photographed by a photographing device such as a camera.

  In recent years, a vehicle display device such as a head-up display device that displays a scene in front of a vehicle by a night vision camera on a display device in the vehicle is becoming popular as one of safe driving means. However, when the head-up display device is used to display the scene in front of the vehicle by the night vision camera, when there are pedestrians or the like under night or heavy fog, There is a problem that a clear image cannot be obtained and is difficult to recognize because the difference in the intensity of infrared rays is not clear. Further, in order to be able to use this night vision camera as an assist device for safe driving, it is necessary to install the display device so that the display surface of the display device always enters the driver's field of view. However, in this case, the scene in front of the vehicle that changes as the vehicle travels is displayed as it is, and the brightness of the displayed image changes drastically, so that the change in the scene always enters the field of view and is bothering the driver. There is a problem of giving.

  In order to solve the above-mentioned problems, a vehicle display device has been devised that aims to provide a device that is easy to understand and does not bother the driver when a vehicle front image is required (Patent Document). 1).

JP 2000-46521 A

  For example, when safe driving support for detecting pedestrians during night driving is performed by a method according to the prior art, if an attempt is made to display an image that reflects the pedestrian detection results, the pedestrian detection time is There is a concern that the person detection frame is shifted from the image displayed on the display device, and an uncomfortable image is displayed. This example is shown in FIG. When a pedestrian 31 moving from right to left is detected as shown in FIG. 3, the pedestrian detection frame 32 is present at the current pedestrian position (31) by the pedestrian before the above calculation processing time. Since it is displayed at the position, the pedestrian detection frame 32 will follow the pedestrian 31 with a delay.

  In the example of Patent Document 1, there is no suggestion or disclosure about a technique for solving the above problem.

  With the above problem as a background, an object of the present invention is to provide a display device for a vehicle that can display a video without a sense of incongruity to a user when displaying a video reflecting a pedestrian detection result.

Means for Solving the Problems and Effects of the Invention

A vehicle display device for solving the above problems includes a photographing device for sequentially photographing the periphery of a vehicle at predetermined photographing time intervals, and a plurality of input image data from the photographing device in the photographing order as a first image data group. The image buffering unit for buffering images and the second image while the input image data is thinned out at a capturing interval longer than the input interval to the image buffering unit in a form branched from the input path to the image buffering unit The first image data group stored in the image analysis unit, which is captured and input as data, performs image analysis for specifying the detection object position on the second image data, and outputs the result, and the image buffering unit Among them, the image corresponding to the second image data is set as the synthesis target image, and the marker image is displayed at the detection target position specified by the image analysis unit with respect to the synthesis target image. The to a marker synthesizing unit to synthesize and output image data, and an image output unit for displaying an output image data, it assumes that comprises a.

  Displaying a plurality of captured image data after buffering means that the image data is displayed delayed by the time stored in the image buffering unit from the time when the image was actually captured. . If a marker image (pedestrian detection frame) is generated in the meantime, the marker image and the image data on which the marker image is generated can be superimposed and displayed, and the marker image follows the detection object (pedestrian). Can be displayed. In addition, since this display delay is at most several hundreds msec, there is no problem in driving the vehicle. With the above-described configuration, it is possible to show the user a video that does not feel strange.

The image buffering unit in the vehicle display device of the present invention has a FIFO image memory for sequentially storing a plurality of first image data by a first-in first-out method, and the first image data is stored in the FIFO image memory. The first image data in the FIFO image memory so that the delay time from input to output corresponds to the time required for image analysis of the second image data corresponding to the first image data. Is stored.

  The time for generating the marker image depends on the system and hardware. Further, the time for reading / writing image data to / from the image data storage means is also known in advance. As a result, the storage capacity can also be calculated in advance. With the above configuration, an image buffering unit (storage medium) having performance and capacity suitable for the system can be selected, and an increase in cost can be suppressed. Further, for example, an identification number is assigned to the captured image data, the image data and the identification number are stored in association with each other, and the marker image and the image data on which the marker image is generated based on the identification number are stored. Compared with the method of displaying together, the processing is not complicated and the processing time can be shortened.

  In the vehicle display device of the present invention, the analog input video signal and the input synchronization signal from the imaging device are distributed and input to the image analysis unit and the image buffering unit, and the image buffering unit receives the input synchronization signal. Each time it arrives, the analog input video signal is sequentially captured as the first image data. The image analysis unit receives the input synchronization signal, captures the second image data, and then completes the image analysis related to the second image data. Until then, the image capturing corresponding to the input synchronization signal that arrives thereafter is paused.

  With the above configuration, when the photographing apparatus outputs an analog input video signal, the analog input video signal can be captured as image data. In addition, it is possible to prevent the next image data from being taken in before the image analysis is completed and hindering the image analysis processing.

  In the vehicular display device according to the present invention, the input image data from the imaging device is digital image data accompanied by identification data for identifying a frame of the input image data. Each time data arrives, it is sequentially captured as first image data. After the image analysis unit captures the second image data, it arrives after the image analysis related to the second image data is completed. The digital image data acquisition is configured to pause.

  With the above configuration, when the photographing apparatus outputs digital image data, the digital image data can be captured as image data for each frame. In addition, it is possible to prevent the next image data from being taken in before the image analysis is completed and hindering the image analysis processing.

  Embodiments of a display device for a vehicle according to the present invention will be described below with reference to the drawings. In FIG. 1, the structure of the display apparatus 1 for vehicles is shown. The vehicle display device 1 includes a camera 11, an image processing IC 12, a detection result video drawing IC 13, a PLD (Programmable Logic Device) 14, a RAM 15, a video synthesis IC 16, and a display 17.

  The camera 11 is configured as a normal CCD camera or a night vision camera, and is attached, for example, in the front grill or in the front center of the ceiling in the passenger compartment so that the front of the vehicle can be photographed. The camera 11 corresponds to the photographing apparatus of the present invention.

  The image processing IC 12 performs predetermined image processing, such as filtering or binarization processing, on the captured image data (second image data: visible light image or infrared image), and an image composed of two-dimensionally arranged pixels. Processing image data is generated. Then, a predetermined recognition process using a pedestrian or an object on the road as a detection target is performed, and an image of the detection target is extracted based on the recognition result. Specifically, the degree of similarity with the reference image is calculated in comparison with the reference image of the detection target stored in advance in the image processing IC 12, and the detection is performed when the degree of similarity exceeds a predetermined threshold. It is determined that the image is an object. The image processing IC 12 corresponds to the image analysis unit of the present invention.

  The detection result video drawing IC 13 calculates the position (coordinates) in the original image data (second image data) of the image of the detection target based on the image processing result in the image processing IC 12. And the marker image according to the magnitude | size of the image of a detection target object is produced | generated. The detection result video drawing IC 13 corresponds to the image analysis unit of the present invention.

  FIG. 2 shows a configuration of a PLD (Programmable Logic Device) 14. The PLD 14 includes a video input unit 141, a timing control 142, a RAM access control unit 143, and a video output unit 144. The PLD 14 temporarily stores the input image data captured by the camera 11 in the RAM 15 via the video input unit 141 and the RAM access control unit 143 as first image data. The first image data read from the RAM 15 is output to the video composition IC 16 as an output video signal together with the output synchronization signal generated by the timing control 142. In addition, reading and writing of image data in the RAM 15 is performed by a first-in / first-out method called FIFO (first-in, first-out). The RAM 15 corresponds to the image buffering unit of the present invention.

  The input synchronization signal is for identifying a frame of image data. When the input image data from the camera 11 is an analog input video signal, for example, a known vertical synchronization signal (Vsync) is used. When the input image data from the camera 11 is digital image data accompanied by identification data for identifying a frame, this identification data is used as an input synchronization signal.

  In addition to the PLD 14, an FPGA (Field Programmable Gate Array: large-scale integrated circuit rewritable in the field), ASIC (Application Specific Integrated Circuit: integrated circuit for specific applications) may be used as long as the function equivalent to the above can be realized. ), DSP (Digital Signal Processor), MPU (Microprocessor Unit), or gate array may be used.

  Returning to FIG. 1, the video composition IC 16 performs, for example, “chroma key” which is a method of transmitting a specific color in the video, or “luminance key” which is a method of transmitting a portion having a specific luminance in the video. Using the marker image created and output from the detection result video drawing IC 13 and the output video signal (first image data) output from the PLD 14, output image data to be displayed on the display 17 is used. Create and output to display 17. The video composition IC 16 corresponds to the marker composition unit of the present invention. The display 17 corresponds to the image output unit of the present invention.

  The display 17 may be any of an image display device such as a liquid crystal display device, an organic electroluminescence display device, a plasma display device, and a field emission display device and a display device equipped with a projection display mechanism by a projector.

  Details of the operation of the vehicle display device 1 will be described with reference to FIG. The camera 11 captures input image data in the order of frame 1, frame 2,..., Frame N,. Then, the image data of frame 1 is input to the video input unit 141 of the PLD 14. The video input unit 141 decodes the data to be stored in the RAM 15. At the same time, an input synchronization signal for identifying the image data of frame 1 is input to the timing control unit 142 of the PLD 14. At this time, the timing control unit 142 increments the counter value of the RAM access counter by one.

  The RAM access control unit 143 determines an address for storing the first image data (decoded) input to the video input unit 141 in the RAM 15 based on the counter value of the RAM access counter calculated above. FIG. 5 shows a storage example of the first image data in the RAM 15. In the example of FIG. 5, the image data storage area is stored in the area indicated by the head address + counter value.

  At the same time, the image data stored previously by the predetermined number of frames is read and output to the video output unit 144. For example, a value obtained by subtracting a predetermined value from the counter value described above is added to the top address of the image data storage area as a read address. The read image data is encoded into an output video signal and output to the video synthesis IC 16. At this time, an output synchronization signal for identifying a frame of the output video signal is also generated and output to the video synthesis IC 16. The

  On the other hand, the image data of frame 1 is input to the PLD 14 and simultaneously taken into the image processing IC 12 as second image data, and the above-described image processing is performed. Then, an image of an object to be detected such as a pedestrian is extracted, the result is output to the detection result video drawing IC 13, and the marker image and its position information are created by the detection result video drawing IC 13 as described above, and video synthesis is performed. It is output to IC16.

  The delay time from when the first image data is input to the RAM 15 until it is output is determined by the specifications of the RAM 15. Further, the processing time in the image processing IC 12 and the detection result video drawing IC 13 can be obtained by measurement in advance. It is also obtained from the specifications of each IC. Furthermore, the number of frames per unit time taken by the camera 11 is also known. Therefore, the difference between the time from when the input image data is captured by the camera 11 and output from the PLD 14 to the time from when the marker image based on the image data is output from the detection result video drawing IC 13 (calculation processing time). Can be calculated as the number of frames. Therefore, an area of the RAM 15 may be secured so that at least the number of frames can be buffered.

  As a result, the image (output video) displayed on the display 17 is displayed with a delay of N frames from the time when the image was taken by the camera 11. For example, the image data of frame 1 is displayed on the display 17 when the image data of frame N is captured by the camera 11. Since a marker image based on the image data of frame 1 is generated during the display delay time, the marker image is synthesized with the image data of frame 1 and displayed. In this case, in FIG. 3, the pedestrian detection frame 32 is displayed so as to surround the pedestrian 31.

  Further, when image analysis based on the image data of frame 1 is being performed, if image data of frame 2 to frame N-1 is input from the camera 11, it is written in the RAM 15 but is captured in the image processing IC 12. (Image analysis cannot be performed even if imported.)

  When image data of frame N is input from the camera 11, the image processing IC 12 performs image analysis based on the image data of frame N.

  In the period from when the image data of frame 1 after image composition (image data for output) is displayed until the image data of frame N after image composition is displayed, the image displayed on the display 17 includes a marker. Images are not superimposed. However, it is possible to avoid a phenomenon in which the marker image appears discontinuous due to the afterimage effect.

  Further, by setting a long delay time from the input of the first image data to the output to the RAM 15 described above, the marker image is, for example, 3 frames more than the image data on which the marker image is created. It can also be displayed in combination with the previous image data. For example, in FIG. 4, a marker image based on the image data of frame 4 is combined with the image data of frame 1 and displayed. Thereby, as shown in FIG. 6, the pedestrian detection frame 32 is displayed on the display 17 so that it slightly precedes the pedestrian 31, and it appears as if the movement direction of the pedestrian is predicted. Can do. Thereby, a driver | operator's response can also be encouraged earlier.

  Although the embodiments of the present invention have been described above, these are merely examples, and the present invention is not limited to these embodiments, and the knowledge of those skilled in the art can be used without departing from the spirit of the claims. Various modifications based on this are possible.

The figure which shows the structure of the display apparatus for vehicles. The figure which shows the structure of PLD. The figure which shows the example of a display by the structure of a prior art. The figure which shows the relationship between an input image and an output image. The figure which shows the structural example of the image data storage area of RAM. The figure which shows the example of a display by the structure of this invention.

Explanation of symbols

1 vehicle display device 11 camera (photographing device)
12 Image processing IC (image analysis unit)
13 Detection result video drawing IC (image analysis unit)
14 PLD
141 video input unit 142 timing control unit 143 RAM access control unit 144 video output unit 15 RAM (image buffering unit)
16 Video composition IC (marker composition unit)
17 Display (image output unit)

Claims (3)

A photographing device that sequentially photographs the periphery of the vehicle at predetermined photographing time intervals;
An image buffering unit for buffering a plurality of input image data from the imaging device in the order of imaging as a first image data group;
The input image data is captured and input as second image data while being thinned out at a capturing interval longer than the input interval to the image buffering unit in a form branched from the input path to the image buffering unit, An image analysis unit that performs image analysis for specifying a detection target position on the second image data and outputs the result;
Of the first image data group stored in the image buffering unit, the one corresponding to the second image data is set as a synthesis target image, and the detection specified by the image analysis unit with respect to the synthesis target image A marker synthesizer that synthesizes a marker image at the object position and sets it as output image data;
An image output unit for displaying and outputting the output image data;
Equipped with a,
The image buffering unit includes a FIFO image memory that sequentially stores a plurality of pieces of the first image data by a first-in first-out method.
The delay time from when the first image data is input to the FIFO image memory until it is output corresponds to the time required for the image analysis of the second image data corresponding to the first image data. Thus, the number of stored first image data in the FIFO image memory is determined . An analog input video signal and an input synchronization signal from the photographing apparatus are distributed and input to the image analysis unit and the image buffering unit, and the image buffering unit receives the analog signal every time the input synchronization signal arrives. The input video signal is sequentially captured as the first image data, and the image analysis unit receives the input synchronization signal, captures the second image data, and then completes the image analysis related to the second image data. 2. The vehicle display device according to claim 1, wherein the image capturing corresponding to the input synchronization signal that arrives thereafter is paused until the operation is performed . The input image data from the photographing device is digital image data accompanied by identification data for identifying a frame of the input image data, and the image buffering unit receives this data every time the input image data arrives. The first image data is sequentially captured, and the image analysis unit captures the second image data, and then completes the image analysis related to the second image data, and then comes digital image data. The vehicle display device according to claim 1, wherein the taking-in of the vehicle is suspended .

Images