Classifications

• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/50—Context or environment of the image
  • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
  • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only
  • H04N23/21—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only from near infrared [NIR] radiation
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
  • B60R2300/106—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using night vision cameras
• • B—PERFORMING OPERATIONS; TRANSPORTING
  • B60—VEHICLES IN GENERAL
  • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
  • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
  • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
  • B60R2300/8033—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for pedestrian protection
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
  • G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
• • G—PHYSICS
  • G02—OPTICS
  • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
  • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
  • G02B27/01—Head-up displays
  • G02B27/0101—Head-up displays characterised by optical features
  • G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems

Definitions

• the invention relates to an on-vehicle night vision device and on-vehicle night vision method for capturing an image of the vicinity of a vehicle in the dark.
• the night vision devices for capturing and displaying an image of the vicinity of a vehicle in the dark, including an image of a view ahead of the vehicle, to ensure the safety of the vehicle while running in the nighttime.
• the night vision devices detect an object, such as an oncoming vehicle, a pedestrian, and a sign around the road, from the captured image of the vicinity of the vehicle to display the captured image and the detected object on a display device.
• Such image capture devices include a vicinity display device for a vehicle disclosed in Japanese Patent Application Publication No. 2004-364112 (JP-A- 2004-364112).
• This vicinity display device for a vehicle displays a visual image from an infrared camera on a display device, and detects a pedestrian or the like in the visual image. The detected pedestrian or the like is enhanced and displayed on the display device.
• an object such as a pedestrian is detected from the captured image, and displayed along with the captured image.
• an image easily viewable to the driver or other person is occasionally not easy to detect an object from, while an image easy to detect an object from is occasionally not easily viewable to the driver or other person. Therefore, for an image captured by the image capture device that is easily viewable to the driver or other person, it may be difficult to detect an object such as a pedestrian from the image, and to accurately display the object.
• an object of the invention is to provide an on-vehicle night vision device and an on-vehicle night vision method that can present an image captured in the dark in a manner easily viewable to the driver, and that can precisely detect an object such as a pedestrian in the image.
• a first aspect of the invention provides an on-vehicle night vision device characterized by including: image capture means for capturing an image of a vicinity of a vehicle in a dark; in-vicinity-of-vehicle object detection means for detecting an object in the vicinity of the vehicle from the captured image by the image capture means; display image generation means for generating a display image based on the captured image; image display means for displaying the display image generated by the display image generation means; image capture condition dividing means for dividing image capture conditions of the image capture means into a first conditions for in-vicinity-of-vehicle object detection and a second conditions for an image to be displayed; and image distribution means for classifying the captured image as one of an image for in-vicinity-of-vehicle object detection and the image to be displayed.
• the image capture conditions of the image capture means are divided into a first conditions for in-vicinity-of-vehicle object detection and a second conditions for an image to be displayed, and images captured under these conditions are classified by the image distribution means. Therefore, conditions suitable for the specific purpose can be set when capturing one of an image for in-vicinity-of-vehicle object detection and the image to be displayed. Thus, it is possible to present an image captured in the dark in a manner easily viewable to the driver, and to precisely detect an object such as a pedestrian in the image.
• the display image generation means may perform a delay process to delay a timing of displaying the image which was classified as the image to be displayed by a time equivalent to a time for the in-vicinity-of-vehicle object detection means to detect an object in the vicinity of the vehicle, and generates the display image by indicating a position of the object in the vicinity of the vehicle detected by the in-vicinity-of-vehicle object detection means in the image to be displayed subjected to the delay process.
• the on-vehicle night vision device may further include identification signal imparting means for imparting an identification signal to the captured image in accordance with the image capture conditions, and the image distribution means may classify the captured image as one of the image for in-vicinity-of-vehicle object detection and the image to be displayed according to the identification signal imparted to the captured image.
• the image distribution means may classify the captured image as one of the image for in-vicinity-of-vehicle object detection and the image to be displayed by image processing of the captured image.
• the on-vehicle night vision device may further include: irradiation means for irradiating the vicinity of the vehicle; and irradiation condition dividing means for dividing irradiation conditions of the irradiation means into a first conditions for in-vicinity-of-vehicle object detection and a second conditions for the image to be displayed.
• a second aspect of the invention provides an on-vehicle night vision method characterized by including: dividing image capture conditions under which to capture an image of a vicinity of a vehicle in a dark, into a first conditions for in-vicinity-of-vehicle object detection and a second conditions for an image to be displayed; classifying the image captured in accordance with the image capture conditions, as one of an image for in-vicinity-of-vehicle object detection and the image to be displayed; detecting an object in the vicinity of the vehicle, from the image for in-vicinity-of-vehicle object detection, generating a display image such that a position of the detected object in the vicinity of the vehicle is indicated the image to be displayed; and displaying the generated display
• the on-vehicle night vision device and method according to the invention it is possible to present an image captured in darkness in a manner easily viewable to the driver, and to precisely detect an object such as a pedestrian in the image.
• FIG. 1 is a block diagram showing the configuration of a night vision system in accordance with an embodiment of the invention
• FIG. 2 is a flowchart showing the procedure of a process performed by a pedestrian detection section
• FIG. 3 is a flowchart showing the procedure of a process following that of FIG. 2; and FIG. 4 is a diagram showing the contents of a visual image signal.
• FIG. 1 is a block diagram showing the configuration of a night vision system as the on-vehicle night vision device in accordance with an embodiment of the invention.
• the night vision system in accordance with this embodiment includes a night vision ECU 1.
• the night vision ECU 1 is a computer of an automobile device to be electronically controlled, and composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a main timer, an input/output interface, and so forth.
• the night vision ECU 1 includes a pedestrian detection section 11, and a drawing section 12 as the display image generation means.
• the pedestrian detection section 11 includes an A/D conversion section 21, a visual image distribution processing section 22 as the image distribution means, a visual image delay processing section 23 as the display image generation means, a pedestrian detection processing section 24 as the in-vicinity-of-vehicle object detection means, and a control section 25.
• the drawing section 12 includes a pedestrian detection frame activation indicator drawing section 31.
• the control section 25 constitutes the image capture condition dividing means and the irradiation condition dividing means.
• To the night vision ECU 1 are connected a near infrared camera 2 as the image capture means, a near infrared emission device 3 as the irradiation means, and a monitor 4 as the image display means.
• the near infrared camera 2 is provided inside the inner rearview mirror of the vehicle, for example, to capture an image of the view ahead of the vehicle.
• the near infrared camera 2 outputs the captured image to the A/D conversion section 21 of the night vision ECU 1 as a visual image signal.
• the near infrared emission device 3 is provided at left and right positions of the front bumper of the vehicle to irradiate the area to be photographed by the near infrared camera 2 ahead of the vehicle.
• the near infrared camera 2 captures an image of an object in the vicinity of the vehicle by receiving light emitted from the near infrared emission device 3 and reflected from the object.
• the near infrared camera 2 is provided with a camera controller, while the near infrared emission device 3 is provided with an emitter controller, so that their image capture conditions and operating conditions (irradiation conditions) can be adjusted (divided) based on an instruction signal output from the control section 25.
• image capture conditions of the near infrared camera 2 and the operating conditions of the near infrared emission device 3 there are respectively provided two types of conditions: those suitable to detect an object in the vicinity of the vehicle, and those suitable to display a captured image.
• the image capture conditions of the near infrared camera 2 include, for example, dividing of internal parameters and the number of shutter releases.
• the operating conditions of the near infrared emission device 3 include, for example, switching between the use of a filter that passes components of a particular wavelength and the use of a filter that cuts them, the emission angle, and whether or not near infrared light is emitted.
• the monitor 4 is provided at a position in the cabin viewable to the driver to display an image output from the drawing section 12 of the night vision ECU 1. While driving the vehicle in the nighttime, the driver can check the vicinity of the vehicle to ensure safety by viewing the vicinity of the vehicle in the dark displayed on the monitor 4.
• the AfD conversion section 21 provided in the pedestrian detection section
• the night vision ECU 1 converts an analog image obtained from the visual image signal output from the near infrared camera 2 into a digital image by AfD conversion.
• the AfD conversion section 21 outputs the digital image obtained by the conversion to the visual image distribution processing section 22.
• the visual image distribution processing section 22 detects a visual image identification signal imparted to the digital image output from the A/D conversion section 21 to classify the output digital image as one of an image for visual image display as the image to be displayed, and an image for pedestrian detection as the image for in-vicinity-of-vehicle object detection, based on the imparted visual image identification signal.
• the output image is classified as an image for visual image display in the case where the visual image identification signal imparted to the output image is a visual image display signal, while the output image is classified as an image for pedestrian detection in the case where the visual image identification signal is an object detection signal.
• the visual image distribution processing section 22 outputs the image for visual image display to the visual image delay processing section 23, and outputs the image for pedestrian detection to the pedestrian detection processing section 24.
• the visual image delay processing section 23 performs a delay process to delay the timing of displaying the image for visual image display output from the visual image distribution processing section 22 by a predetermined unit time tl.
• the unit time tl is set to be equivalent to the time for the pedestrian detection processing section 24 to detect a pedestrian from the image for pedestrian detection.
• the visual image delay processing section 23 outputs the display image subjected to the delay process to the pedestrian detection frame activation indicator drawing section 31 of the drawing section 12.
• the pedestrian detection processing section 24 detects a pedestrian from the image for pedestrian detection output from the visual image distribution processing section 22. Upon detection of a pedestrian, the pedestrian detection processing section 24 outputs the position of the pedestrian in the image for pedestrian detection to the pedestrian detection frame activation indicator drawing section 31 of the drawing section 12.
• the control section 25 provides two modes: the image recognition mode and the visual image display mode.
• An instruction signal on the camera parameters of the near infrared camera 2 and the operating conditions of the near infrared emission device 3 in accordance with the present mode is output to the near infrared camera 2 and the near infrared emission device 3.
• the camera controller of the near infrared camera 2 imparts a synchronization signal to the visual image signal to be output to the pedestrian detection section 11.
• the near infrared camera 2 (camera controller) constitutes the identification signal imparting means.
• the camera controller imparts a visual image identification signal in accordance with the present mode to the header portion immediately after the synchronization signal.
• the interval at which the synchronization signal is imparted is shorter than the unit time tl, and several synchronization signals are imparted in each unit time tl.
• the pedestrian detection frame activation indicator drawing section 31 of the drawing section 12 generates a display image obtained by indicating the position of the pedestrian output from the pedestrian detection processing section 24 in the image for visual image display output from the visual image delay processing section 23.
• the pedestrian detection frame activation indicator drawing section 31 outputs the generated display image to the monitor 4.
• a headlight switch (not shown) for turning on and off the headlight of the vehicle is connected to the night vision ECU 1. Turning on the headlight switch to turn on the headlight starts the night vision ECU 1.
• FIGs. 2 and 3 are flowcharts showing the procedure of a process performed by the pedestrian detection section 11 of the night vision system.
• the pedestrian detection section 11 determines whether or not the night vision system is activated with the night vision ECU 1 started (Sl). If it is determined that the night vision system is not activated, camera control by the control section 25 is stopped and the main timer of the night vision ECU 1 is stopped (S2), before the process returns to step Sl to repeat the same process.
• step S3 If it is determined that the night vision system is activated, it is determined whether or not the operation of the near infrared camera 2 is under the control of the control section 25 (S3). If it is determined that the operation of the near infrared camera 2 is not under the control of the control section 25, the control section 25 starts controlling the near infrared camera 2 (S4), before the process returns to step S3. [0032] If it is determined that the operation of the near infrared camera 2 is under the control of the control section 25, it is determined whether or not the main timer of the night vision ECU 1 is in operation (S5). If it is determined that the main timer of the night vision ECU 1 is not in operation, the main timer of the night vision ECU 1 is started (S6), before the process returns to step S5.
• step S7 it is determined whether or not the operating conditions of the near infrared emission device 3 are met. If it is determined that the operating conditions of the near infrared emission device 3 are met, it is determined whether or not the near infrared emission device 3 is turned on (S8). If it is determined that the near infrared emission device 3 is not turned on, the near infrared emission device 3 is turned on (S9). If the near infrared emission device 3 is determined to be turned on, the process proceeds to step S 12 (see FIG 3) without any change.
• the near infrared emission device 3 is turned on (SlO). If it is determined that the near infrared emission device 3 is turned on, the near infrared emission device 3 is turned off (SIl). If the near infrared emission device 3 is determined to be not turned on, the process proceeds to step S12 (see FIG. 3) without any change.
• step S12 it is determined whether or not the main timer of the night vision ECU 1 is indicating an integer n times the unit time tl (S12). If it is determined that the main timer of the night vision ECU 1 is indicating the integer n times the unit time tl, the control section 25 is set to the image recognition mode, and set to the visual image retrieval state (S 13). In the image retrieval state, images can be retrieved from the visual image distribution processing section 22 to the pedestrian detection processing section 24. If it is determined that the main timer of the night vision ECU 1 is not indicating the integer n times the unit time tl, the process proceeds to step S 14 without any change.
• step S14 it is determined whether or not retrieval of visual images to the pedestrian detection section 11 have been completed. If it is determined that retrieval of visual images have been completed, the control section 25 is set to the visual image display mode, and the visual image retrieval state from the near infrared camera 2 is canceled (S 15). If it is determined that retrieval of visual images have not been completed, the process proceeds to step S 16 without any change, until retrieval of visual images have been completed.
• the near infrared camera 2 outputs the captured image to the pedestrian detection section 11 as a visual image signal.
• the camera controller of the near infrared camera 2 generates a visual image signal based on the captured image.
• This visual image signal includes a visual image identification signal AS in the header portion following a synchronization signal SS, as shown in FIG. 4.
• the header portion is followed by visual image data VD.
• the visual image identification signal AS an object detection signal is recorded when the image is captured under conditions suitable to detect an object in the vicinity of the vehicle, and a visual image display signal is recorded when the image is captured under conditions suitable to display a captured image.
• the previous mode is stored as the current mode (S 18), before the process by the pedestrian detection section 11 is terminated.
• the A/D conversion section 21 converts the visual image signal output from the near infrared camera 2 into a digital image
• the visual image distribution processing section 22 classifies the converted digital image as one of an image for visual image display and an image for pedestrian detection (S 19).
• the visual image distribution processing section 22 verifies whether the visual image identification signal imparted to the header portion of the visual image signal output from the near infrared camera 2 is one of a visual image display signal and an object detection signal. If the imparted visual image identification signal is a visual image display signal, the image is output to the visual image delay processing section 23. If the imparted visual image identification signal is an object detection signal, the image is output to the pedestrian detection processing section 24. Specifically, with reference to the example shown in FIG. 4, in the case where four synchronization signals are imparted in each unit time tl, for example, three visual image display signals VAS and one object detection signal OAS are imparted as the visual image identification signals while four synchronization signals are imparted. [0039] After the lapse of the time tl for the pedestrian detection processing section
• the visual image delay processing section 23 outputs a display image obtained by performing a delay process on the image output from the visual image distribution processing section 22 to the pedestrian detection frame activation indicator drawing section 31 of the drawing section 12 (S20).
• the pedestrian detection processing section 24 detects a pedestrian or the like captured in the image output from the visual image distribution processing section 22 by image processing. If a pedestrian or the like is detected, a pedestrian detection frame is output to the pedestrian detection frame activation indicator drawing section 31 of the drawing section 12 to emphatically indicate the existence of the pedestrian or the like by displaying a detection frame at the corresponding position (S20). Then, the process by the pedestrian detection section 11 is terminated.
• the pedestrian detection frame activation indicator drawing section 31 of the drawing section 12 When the process by the pedestrian detection section 11 is finished, the pedestrian detection frame activation indicator drawing section 31 of the drawing section 12 generates an image by adding the pedestrian detection frame to the image output from the visual image delay processing section 23 at the position output from the pedestrian detection processing section 24 to display the generated image on the monitor 4.
• the near infrared camera 2 captures an image of the vicinity of the vehicle in darkness, and a visual image signal based on the captured visual image is output from the near infrared camera 2 to the pedestrian detection section 11 of the night vision ECU 1.
• a visual image identification signal has been imparted to the visual image signal output from the near infrared camera 2 to the pedestrian detection section 11 of the night vision ECU 1.
• the visual image distribution processing section 22 of the pedestrian detection section 11 verifies the visual image identification signal imparted to the visual image signal to distribute the output one of visual image signal to the visual image delay processing section 23 and the pedestrian detection processing section 24.
• the control section 25 outputs to the near infrared camera 2 an instruction signal to divide the image capture conditions of the near infrared camera 2.
• the near infrared camera 2 divides the image capture conditions based on the instruction signal output from the control section 25.
• those containing a visual image captured under conditions suitable to detect an object in the vicinity of the vehicle have been imparted with an object detection signal
• those containing a visual image captured under conditions suitable to display a captured image have been imparted with a visual image display signal.
• the visual image delay processing section 23 receives an image for visual image display captured under conditions suitable to display a captured image
• the pedestrian detection processing section 24 receives an image for pedestrian detection captured under conditions suitable to detect an object in the vicinity of the vehicle.
• the visual image distribution processing section 22 can classify images as one of those for pedestrian detection and those for visual image display by just verifying the identification signal. Thus, it is easy to classify images.
• control section 25 outputs to the near infrared emission device 3 an instruction signal to divide the operating conditions. Therefore, the operating conditions of the near infrared emission device 3 can be set to one of those suitable to detect an object in the vicinity of the vehicle and those suitable to display a captured image. Thus, it is possible to capture more suitable images for in-vicinity-of-vehicle object detection and for the image to be displayed. [0045] Although a preferred embodiment of the invention has been described above, the invention is not limited to the above embodiment.
• a visual image identification signal has been imparted to a visual image signal to classify an image as one of an image for pedestrian detection and an image to be displayed in the above embodiment
• an image may be classified by image processing of the output visual image signal or the like.
• a pedestrian is treated as an object in the vicinity of the vehicle in the above embodiment
• other objects such as a traffic signal and a sign in the vicinity of the road may be treated as an object.

Landscapes

• Physics & Mathematics (AREA)
• Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Theoretical Computer Science (AREA)
• Optics & Photonics (AREA)
• Health & Medical Sciences (AREA)
• Toxicology (AREA)
• Signal Processing (AREA)
• Closed-Circuit Television Systems (AREA)
• Traffic Control Systems (AREA)

Applications Claiming Priority (2)

Publications (2)

Family

ID=39345306

Family Applications (1)

Country Status (2)

Cited By (2)

Citations (1)

Family Cites Families (3)

• 2006
  • 2006-12-21 JP JP2006344658A patent/JP2008160305A/ja active Pending
• 2007
  • 2007-12-19 WO PCT/IB2007/004274 patent/WO2008078187A2/en active Application Filing

Patent Citations (1)

Also Published As

Similar Documents

Legal Events