JP2011234031A - On-vehicle imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an on-vehicle imaging apparatus that properly supports the driver's drive by displaying a risk factor existing on a road without preventing the display of a photographed video of the surrounding of the vehicle.SOLUTION: An on-vehicle imaging apparatus has: an imaging optical system; an imaging element that carries out photoelectric conversion of a subject image acquired from the imaging optical system; and a signal processor that carries out a process of an image signal outputted from the imaging element. The signal processor superposes and displays risk prediction regions 26 and 27 where a risk factor actually exists or where the existence of a risk factor is predicted around a vehicle 22, on an imaged image depending on a running condition of the vehicle 22, using a transparent line at least one of whose color, concentration, and width is changed or a mask.

Description

  The present invention relates to an in-vehicle imaging apparatus capable of displaying an image around a vehicle photographed by an imaging means such as an in-vehicle camera on an in-vehicle monitor.

  In this type of in-vehicle image pickup device, an image of the periphery of the vehicle imaged by the image pickup means is displayed on a monitor installed in the vehicle, so that a driver who visually recognizes the monitor can sensibly recognize the running state of the vehicle. There is something. At this time, the display on the monitor displays the course of the vehicle predicted from the current or past driving state of the vehicle (hereinafter referred to as “running predicted trajectory” as appropriate) in addition to the image around the vehicle. Some support the driver's driving.

  For example, Patent Literature 1 discloses a technique for displaying a predicted traveling locus of a vehicle in a superimposed manner on an image captured from a back view camera when the vehicle is moved backward for parking. Patent Document 2 discloses a technique for calculating a predicted travel trajectory based on a steering angle detected by a steering angle sensor installed in the steering of a vehicle.

Japanese Laid-Open Patent Publication No. 2004-262449 JP 11-334470

  However, in the above-described Patent Documents 1 and 2, the predicted traveling locus is superimposed and displayed as a solid solid line on the image around the vehicle photographed by the imaging means. Therefore, there is a technical problem in that a portion of the video around the vehicle that overlaps the predicted traveling locus is hidden by the predicted traveling locus displayed by a solid solid line, and information about the portion is lost. In particular, when the lost information is important information for the driver to drive the vehicle safely (for example, pedestrians, other traveling vehicles or obstacles), this problem becomes even more serious. It becomes.

  As a supplementary explanation, it is necessary for a traveling driver to accurately grasp a risk factor that is actually present on the road or expected to be present, such as a pedestrian on the road, another traveling vehicle, or an obstacle. is there. In the above-described background art, although the driving safety is achieved by displaying the predicted traveling locus, no consideration is given to such a risk factor when displaying the predicted traveling locus.

  The present invention has been made in view of, for example, the above-described problems, and appropriately supports driving of a driver by displaying a risk factor existing on the road without disturbing display of a video around the photographed vehicle. It is an object of the present invention to provide an on-vehicle imaging device that can be used.

  In order to solve the above problems, an image pickup apparatus for a vehicle according to the present invention picks up an image pickup optical system for picking up the periphery of the vehicle, an image pickup element for photoelectrically converting a subject image obtained via the image pickup optical system, and an output from the image pickup element. A signal processing unit for processing an image signal to be processed, wherein the signal processing unit is configured to determine a risk prediction region in which a risk factor is actually present or predicted to exist in the vicinity of the vehicle according to a running state of the vehicle. Then, the image is superimposed and displayed on the captured image using a transparent line or mask in which at least one of color, density and width is changed.

  In one aspect of the vehicle imaging apparatus of the present invention, the traveling state is a distance or a vehicle speed from the vehicle to the danger prediction area.

  In another aspect of the vehicle image pickup device of the present invention, the danger prediction area includes the predicted travel path of the vehicle, an obstacle area where an obstacle exists on or around the predicted travel path, and a stop target space of the vehicle. , At least one of a door opening / closing region required for opening / closing a door included in the vehicle when the vehicle is stopped in the stop target space, and a vehicle peripheral region to which the driver of the vehicle should always pay attention. And

  In another aspect of the imaging apparatus for a vehicle of the present invention, the signal processing unit displays the mask so as to surround the danger prediction area when the danger prediction area is displayed using the mask. And

  In another aspect of the vehicle imaging apparatus of the present invention, the signal processing unit displays the transmittance of the transmissive line or the mask as the distance from the vehicle to the danger prediction area increases. It is characterized by doing.

  In another aspect of the vehicle imaging device of the present invention, when the signal processing unit displays the danger prediction area using the line, the distance from the vehicle to the danger prediction area increases as the distance increases. The display is characterized in that the line thickness is reduced.

  According to the present invention, the risk prediction region in which the risk factor is actually present or predicted to be present is superimposed and displayed on the image captured using the transmissive line or mask according to the running state of the vehicle. Therefore, it is possible to display the risk factors existing on the road without disturbing the display of the image around the photographed vehicle, and to appropriately support the driving of the driver. In particular, by changing at least one of the color, density, and width of the danger prediction area according to the driving state of the vehicle, the driver does not feel uncomfortable in the captured image (that is, according to the driver's feeling). The risk prediction area can be displayed in a superimposed manner.

It is a block diagram which shows the structure of the imaging device which concerns on a present Example, and its peripheral device. It is a schematic diagram which shows typically arrangement | positioning of the imaging system in a vehicle from a vehicle upper viewpoint. It is a flowchart figure which shows operation | movement of the imaging device which concerns on a present Example, and its peripheral device. It is the schematic diagram which represented typically the actual driving | running | working estimated locus | trajectory of the vehicle of a present Example from the viewpoint from the vehicle upper direction. It is a schematic diagram which shows typically the driving | running | working estimated locus drawn on a monitor. It is a list which shows the table referred when selecting the display form of a danger prediction area | region. It is a flowchart figure which shows the process for transparently superimposing and displaying a danger prediction area | region on a captured image in steps. It is a schematic diagram which shows typically the specific example of the image by which the danger prediction area | region was superimposed and displayed on the captured image. It is a schematic diagram which shows typically the specific example of the image by which the danger prediction area | region was superimposed and displayed on the captured image. It is a schematic diagram which shows typically the specific example of the image by which the danger prediction area | region was superimposed and displayed on the captured image. It is a schematic diagram which shows typically the specific example of the image by which the danger prediction area | region was superimposed and displayed on the captured image. It is a schematic diagram which shows typically the specific example of the image by which the danger prediction area | region was superimposed and displayed on the captured image. It is a schematic diagram which shows typically the specific example of the image which superimposed and displayed the driving | running | working prediction locus | trajectory as a danger prediction area | region on the captured image acquired from the front view camera.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, relative arrangements, and the like of the components described in this embodiment are not intended to limit the scope of the present invention unless otherwise specified, but are merely illustrative examples. Not too much.

  First, with reference to FIG.1 and FIG.2, the structure of the imaging device which is an example of the imaging device for vehicles of this invention and its peripheral device is demonstrated. Here, FIG. 1 is a block diagram illustrating a configuration of the imaging apparatus and its peripheral devices according to the present embodiment. FIG. 2 is a schematic diagram schematically showing the arrangement of the imaging system in the vehicle from the vehicle upper viewpoint.

  In this embodiment, the imaging system 2, the monitor 12, the processing system 14, the drive system 16, and the output system 18 are provided in the vehicle 22.

  The imaging system 2 is composed of a plurality of cameras capable of acquiring a surrounding image of the vehicle 22. Specifically, as shown in FIG. 2, the imaging system 2 includes four types of cameras, a front view camera 2a, a back view camera 2b, a left side view camera 2c, and a right side view camera 2d, according to the shooting direction. It is configured. In FIG. 2, the outline of the main body of the vehicle 22 is indicated by a dotted line. Each of these cameras included in the imaging system 2 stores a lens 4, an imager 6 that receives the light transmitted through the lens 4 and outputs a captured image, and an image of a risk prediction region that is to be superimposed on the captured image. And a controller 10 that controls the operation of the imager 6 by appropriately transmitting instructions. Note that the memory 8 and the controller 10 may be provided in each of a plurality of cameras constituting the imaging system 2 or may be provided so as to be shared by a plurality of cameras.

  The monitor 12 acquires the image data output from the imager 6 based on a command from the controller 10 and displays the image on a display monitor such as a liquid crystal screen. The processing system 14 processes the captured image obtained from the transmitted light received by the imager 6 via the lens 4 as necessary, and also drives the drive system 16 according to the traveling state of the vehicle 22 determined from the processed image. A command is output to the output system 18 to control these operations. The drive system 16 is a reduction acceleration means such as an accelerator and a brake of the vehicle, for example, and the safety of the traveling state of the vehicle 22 can be ensured by appropriately controlling these. The output system 18 is an informing means for informing the driver of the danger of the traveling state by issuing an alarm when it is determined that the traveling state of the vehicle 22 grasped in the processing system 14 is dangerous.

  Next, with reference to FIG. 3, the operation of the imaging apparatus and its peripheral devices according to the present embodiment will be described in detail. FIG. 3 is a flowchart illustrating the operation of the imaging apparatus and its peripheral devices according to the present embodiment. In the present embodiment, an operation example in which the danger prediction area is superimposed and displayed on the captured image acquired by the back view camera 2b when the vehicle 22 is parked will be described.

  First, the controller 10 determines whether or not the shift lever is set to “reverse” from the drive system 16 (step S101). Here, when the shift lever is not set to “reverse” (step S101: NO), the controller 10 does not need to acquire the rear image of the vehicle 22 by the back view camera 2b, and thus ends the process (END). ). On the other hand, if the shift lever is set to “reverse” (step S101: YES), the controller 10 transmits a command to the imager 6 to acquire a captured image behind the vehicle 22 from the back view camera 2b (step S101). S102).

  Subsequently, the controller 10 determines whether or not it is necessary to display a danger prediction area in the acquired captured image (step S103). The vehicle 22 according to the present embodiment switches between a superimposed display mode in which the danger prediction area is superimposed on the captured image and a normal mode in which the captured image is displayed as it is without being superimposed on the captured image (not illustrated). The controller 10 can determine whether or not it is necessary to display the danger prediction area in the acquired captured image by acquiring the state of the changeover switch.

  When it is determined that it is not necessary to display the danger prediction area (step S103: NO), the controller 10 advances the process to step S107, and the captured image acquired in step S102 is displayed without superimposing the danger prediction area on the captured image. Is displayed on the monitor 12 as it is.

  On the other hand, when it is determined that the danger prediction area needs to be displayed (step S103: YES), the controller 10 detects the danger prediction area based on the captured image acquired in step S102 (step S104). Here, the danger prediction area broadly means an area where a risk factor exists or may exist for the driver of the vehicle 22.

  As an example of the danger prediction area, there is, for example, a predicted travel locus predicted from the current or past travel state. As another example of the danger prediction area, a parking space where the vehicle 22 is stopped at the time of parking (hereinafter referred to as “stop target space” as appropriate), a route from the current position of the vehicle 22 to the stop target space, Moreover, when the vehicle 22 is stopped in the stop target space, an area required for the door to open and close for getting on and off the driver is also included. In addition, about the area | region required for the said door to open and close, you may employ | adopt as an example of a danger prediction area | region similarly not only a driver but the area | region required for a passenger to open and close a door. In this case, the presence / absence of the passenger and the boarding position in the vehicle are determined by, for example, the presence / absence of the seat belt in each seat and the load sensor provided in each seat, and the door corresponding to the passenger's boarding position opens and closes. An area required for the evaluation may be displayed with priority as an example of the danger prediction area. Furthermore, as another example of the danger prediction area, when there is an obstacle on or around the above-described danger prediction area, the existence area of the obstacle is also exemplified. As another example of the danger prediction area, the driver must always pay attention to an area within a certain distance from the vehicle (hereinafter, referred to as “vehicle peripheral area” as appropriate) when driving the vehicle 22 safely. Therefore, the area around the vehicle also corresponds. Like these areas, the risk prediction area in the present invention includes a wide range of areas where a risk factor actually exists or may exist for the driver.

  Further, when detecting the danger prediction area, the traveling state of the vehicle 22 may be taken into consideration. For example, the traveling state of the vehicle 22 includes the speed and position of the vehicle 22, the steering angle detected by a steering angle sensor (not shown) installed in the steering, and the traveling road surface state (for example, road width and road surface). Etc.) are considered.

  Here, with reference to FIG.4 and FIG.5, the detection method of the driving | running | working estimated locus | trajectory which is an example of a danger prediction area | region is demonstrated concretely. FIG. 4 is a schematic diagram schematically showing an actual travel prediction trajectory of the vehicle according to the present embodiment from a viewpoint from above the vehicle. FIG. 5 is a schematic diagram schematically showing a predicted traveling locus drawn on the monitor 12.

  As shown in FIG. 4, in this embodiment, the case where the steering angle of the vehicle 22 is set so that the vehicle 22 goes straight backward is illustrated. In FIG. 4, the upper side is shown as the rear side of the vehicle 22.

  In FIG. 4, the predicted trajectory in the traveling direction of the vehicle (that is, the course prediction guide 24) indicates the predicted traveling trajectory in the traveling direction of the vehicle 22 obtained from the rudder angle detected by the rudder angle sensor of the vehicle 22 with a solid line. A guide for a sense of distance is created by providing a guide in the vehicle width direction for each specified distance a. In this way, the processing system 14 converts to the predicted course prediction guide 25 viewed from the rear of the vehicle as shown in FIG. 5 based on the course prediction guide 24 shown in FIG.

  Returning to FIG. 3, the controller 10 then selects a display form in a case where the danger prediction area detected in step S104 is superimposed on the captured image acquired in step S102 from a preset table (step S105). ). Here, an appropriate display form pattern is set for each type of danger prediction area detected in step S104 in the table, and is stored in advance in a storage device (not shown) provided in the controller 10. In step S105, the controller 10 accesses the storage device to select an appropriate pattern according to the type of the danger prediction area from the table.

  Here, with reference to FIG. 6, the specific contents of the table in which the display mode of the danger prediction area is set will be described. FIG. 6 is a list showing a table that is referred to when selecting the display mode of the danger prediction area.

  In FIG. 6, the color and transmittance are patterned for each type of danger prediction area. For example, an orange line is selected when a predicted travel path is displayed as the risk prediction area. In addition, when an obstacle is displayed as a danger prediction area, a gray mask is selected (exactly, as will be described later, the obstacle is displayed as a mask around the obstacle so that the obstacle is compared with the surrounding area. Displayed by subtracting). In addition, when the stop target space is displayed as the danger prediction area, an orange line is selected. Further, when a door opening / closing area (that is, an area required for opening / closing the door when the vehicle 22 is stopped in the stop target space) is displayed as the danger prediction area, the red filling is selected. Further, when a vehicle peripheral area is displayed as the danger prediction area, a blue line is selected.

  Note that the combinations shown in FIG. 6 are merely examples, and may be set as an appropriate pattern for the driver to recognize ergonomically without a sense of incongruity according to the type of the danger prediction area and the running state of the vehicle. Note that these patterns are not necessarily set in advance, and may be customizable by a driver's operation.

  Returning to FIG. 3 again, the controller 10 selects the display mode of the risk prediction area in step S105, and then instructs the processing system 14 to draw the risk prediction area in a superimposed manner on the captured image acquired in step S102. The created image data is created (step S106). Particularly in the present embodiment, this processing is performed so that the danger prediction area is transparently superimposed on the captured image.

  Here, with reference to FIG. 7, the content of the process performed in step S106 is demonstrated concretely. FIG. 7 is a flowchart showing step-by-step processing for transparently displaying a danger prediction area on a captured image.

  First, the processing system 14 is based on a command from the controller 10, and is an HSL color space having a solid color (that is, a transmittance of 0%) corresponding to the display mode of the danger prediction area selected in step S105 (see FIG. 3). (H1, S1, L1) is acquired (step S201). The HSL color space corresponding to each solid color is stored in advance in a storage device (not shown) provided in the processing system 14, and in step S201, the processing system 14 accesses the storage device. The solid color HSL color space corresponding to the display form of the selected danger prediction area is acquired.

  Subsequently, the processing system 14 acquires the transmittance T when the danger prediction area is superimposed and displayed on the captured image (step S202). Here, the transmittance T is appropriately set as a transmittance that allows the driver to know the danger prediction area reliably and accurately without any sense of incongruity and that does not cause a problem in the safe driving of the driver because the captured image is hidden by the danger prediction area. Is set. For example, the driver tends to recognize dark objects that are visually close and darkly recognize those that are far away. Therefore, if the transmittance T is set so that the distance from the vehicle 22 becomes darker and the lighter the closer, the thinner. Good.

  Further, the transmittance T may be set according to the importance of the danger prediction area. In this case, the transmittance is set to be low (that is, dark) in the case of the risk prediction region that is highly important for the driver, and the transmittance is set to be high (that is, thin) in the case of the risk prediction region that is relatively low in importance. Good.

  In setting the transmittance T, a traveling state such as the speed of the vehicle 22 may be taken into consideration. For example, the display density of the danger prediction area may be increased by setting the transmittance to increase as the speed of the vehicle 22 increases so as to alert the driver.

  The transmittance T thus set is stored in advance in a storage device (not shown) provided in the processing system 14, and in step S202, the processing system 14 accesses the storage device to A transmittance T suitable for superimposed display of the prediction area is acquired.

Subsequently, the processing system 14 uses the HSL color space (H1, S1, L1) acquired in step S201 and the transmittance T acquired in step S202 to determine the HSL color space for actually displaying the danger prediction area. Calculate (step S203). Specifically, as shown in the following equation (1), the HSL when the danger prediction area is actually displayed by multiplying the luminance L1 by the transmittance T in the HSL color space (H1, S1, L1). The color space (H1, S1, L1 ′) is calculated.
L1 ′ = L1 × T (1)

  Subsequently, based on the HSL color space (H1, S1, L1 ′) at the time of actually displaying the danger prediction area calculated in this way, the processing system 14 outputs the danger prediction area superimposed on the captured image. Image data is created (step S204), and the output image data is transmitted to the imager 6 (step S205).

  Returning to FIG. 3 again, the controller 10 then outputs the output image data created by the processing in step S106 to the monitor 12 (step S107). In this way, an image in which a transparent danger prediction area is superimposed and displayed on the captured image is displayed on the monitor 12.

  Particularly in the present embodiment, when such a risk prediction area is drawn in a superimposed manner on a captured image by a line or a mask, these lines or the mask are displayed transparently. Here, “transparently” means that when the danger prediction area is superimposed and displayed on the captured image, the captured image has transparency so that it can be seen through. If the danger prediction area that does not have transparency is displayed non-transparently, the original captured image is partially hidden in the area where the danger prediction area is displayed, and the driver needs that much. Information is lost. In particular, when the lost information is important information for the driver to drive the vehicle safely (for example, pedestrians, other traveling vehicles or obstacles), this problem is serious for safe driving. Will lead to negative results. In contrast, in this embodiment, the danger prediction area is displayed using a transparent line or mask, so that the original captured image is not hidden even in the area where the danger prediction area is displayed. Therefore, appropriate support can be realized so that the driver can drive safely.

  Here, with reference to FIG. 8, a specific example of an image displayed on the monitor 12 in which a transparent danger prediction area is superimposed and displayed on a captured image will be described. FIG. 8 is a schematic diagram schematically illustrating a specific example of an image in which a danger prediction area is superimposed and displayed on a captured image.

  FIG. 8A is an example in which a predicted travel path 26 and a target stop space 27 are superimposed and displayed as a risk prediction area on a captured image acquired from the back view camera 2b. In FIG. 8A, the predicted travel path 26 and the target stop space 27 are shown in a transparent manner, and in order to make the target stop space 27 that the driver will pay the most attention during travel stand out, The transmittance of the stop target space 27 is set to be lower (that is, darker) than the transmittance of the predicted travel path 26. Further, the distance to the driver is given by displaying the line width wider as the vehicle 22 is approached.

  FIG. 8B is an example in which the danger prediction area is displayed so as to be surrounded by the mask 28 in addition to FIG. In this case, since the danger prediction area surrounded by the mask 28 can be emphasized as compared with the surrounding area, the driver's attention can be more accurately directed to the danger prediction area.

  FIG. 8C shows a display example in the case where another vehicle 29 that is an obstacle exists as the danger prediction area. In FIG. 8C, by displaying the surroundings of the other vehicle 29 with a mask, the driver can be informed of the presence.

  FIG. 8D is an example in which a boundary line 30 indicating a vehicle peripheral area is superimposed and displayed as a danger prediction area on the captured image acquired from the back view camera 2b. As described above, the vehicle peripheral area is defined as an area within a certain distance from the vehicle 22 to which the driver should always pay attention. In FIG. 8D, the boundary is a transparent boundary line on the captured image. 30 is superimposed and displayed.

  FIG. 8E shows an example in which a door opening / closing area 31 is displayed in addition to the predicted travel path 26 and the stop target space 27 as the risk prediction area. Thereby, since the area | region required for a door to open and close when stopping the vehicle 22 to the stop target space 27 can be grasped | ascertained beforehand, a driver is conscious of the door open / close area 31, and is suitable in the stop target space 27. The vehicle 22 can be guided to a different place.

  Needless to say, the image in which the transparent danger prediction region is superimposed and displayed on the captured image is not limited to the specific example shown in FIG. 8, and may be a combination thereof.

  Further, in the present embodiment, the case where the danger prediction area is superimposed and displayed on the captured image acquired by the back view camera 2b has been described in detail. It goes without saying that the danger prediction area can be displayed in a superimposed manner on the captured video as well. For example, FIG. 9 is an example in which a predicted travel path 26 is superimposed and displayed as a risk prediction area on the captured image acquired from the front view camera 2a. In FIG. 9, a white line 35 that defines the width of the travel route and a guard rail 36 are shown as images.

  As described above, according to the present embodiment, the danger prediction area is superimposed on the captured image using a semi-transparent line or mask according to the traveling state of the vehicle, thereby It is possible to display the risk factors that the driver needs to grasp for safe driving along with the predicted driving trajectory without disturbing the display of the image around the captured vehicle, so that the driver's driving can be supported appropriately it can.

  The present invention provides, for example, an on-vehicle imaging that allows a driver viewing the monitor to sensuously recognize the running state of the vehicle by displaying an image around the vehicle captured by the imaging means on a monitor installed in the vehicle. It can be used as a device.

2 Imaging System 4 Lens 6 Imager 8 Memory 10 Controller 12 Monitor 22 Vehicle 26 Expected Travel Trajectory 27 Stop Target Space

Claims (6)

An imaging optical system that images the periphery of the vehicle, an imaging element that photoelectrically converts a subject image obtained via the imaging optical system, and a signal processing unit that processes an image signal output from the imaging element,
The signal processing unit transmits a risk prediction region where a risk factor is actually present or is predicted to be present in the vicinity of the vehicle by changing at least one of color, density and width according to the running state of the vehicle. A vehicle-mounted imaging device, wherein the image is superimposed and displayed on the captured image using a sex line or a mask.   The vehicle imaging apparatus according to claim 1, wherein the traveling state is a distance or a vehicle speed from the vehicle to the danger prediction area.   The danger prediction area includes a predicted travel path of the vehicle, an obstacle area where an obstacle exists on or around the predicted travel path, the stop target space of the vehicle, and the stop target space when the vehicle is stopped. The vehicle imaging according to claim 1, wherein the vehicle imaging region is at least one of a door opening / closing region required for opening / closing the door of the vehicle and a vehicle peripheral region to which a driver of the vehicle should always pay attention. apparatus.   The said signal processing part displays the said mask so that the said danger prediction area | region may be displayed, when displaying the said danger prediction area | region using the said mask. The vehicle imaging device described.   5. The display according to claim 1, wherein the signal processing unit displays so that the transmittance of the transmissive line or the mask increases as the distance from the vehicle to the danger prediction area increases. The vehicle imaging device according to claim 1.   When displaying the danger prediction area using the line, the signal processing unit displays the line so that the thickness of the line becomes thinner as the distance from the vehicle to the danger prediction area increases. The imaging device for vehicles according to any one of claims 1 to 5 characterized by things.