JP2009044570A - Vehicle's blind spot assistant system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a "vehicle's blind spot assistant system" which can cover a driver's blind spot region by images by a top view system. <P>SOLUTION: When inputting images captured by a plurality of cameras for photographing the circumference of a vehicle and forming a top view image which is a synthesized image viewed from a virtual point of view, a blind spot region in the circumference of the vehicle viewed from the driver is calculated and then a top view image of a range after move to the side for covering the calculated blind spot region is formed and displayed. At that time, the position of the driver's point of view is detected by the image captured by the camera for photographing the driver or by inputting the data related to the position of the driver's eyes such as the driver's seating height. Moreover, the top view image of the range after move to the side for covering the blind spot region is an image wherein the driver-side corner in a front part of the vehicle in the blind spot region conforms to the driver-side corner in a front part of the vehicle in the top view image, or an image covering the whole blind spot region is formed as the top view image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention provides a vehicle blind spot assist in which a blind spot portion in the vicinity of a vehicle generated when the vehicle is driven can be assisted by photographing the outside with a plurality of cameras provided on the vehicle and displaying an image around the vehicle having a viewpoint at the top of the vehicle. About the system.

  When putting a vehicle in a narrow garage or parking lot, the driver needs to be very careful and sometimes accidents such as rubbing the surrounding vehicle with the surrounding object may occur. Also, when taking a vehicle out of a garage or parking lot, it is necessary to pay close attention to surrounding objects and people. In addition to entering and exiting garages and parking lots, vehicles such as driving on extremely narrow roads or contacting with surrounding objects when passing on opposite roads on narrow roads, or preventing wheels from falling into grooves, etc. There are many cases where sufficient attention is required around the area.

  As countermeasures, the rear of the vehicle, which has been widely installed in vehicles in recent years, is photographed and displayed on a monitor, and a rear-facing camera that facilitates parking and parking, When a vehicle leaves a narrow road with poor visibility, use a camera outside the vehicle, such as a side-view camera, to check the vehicle traveling on the other road, and monitor the situation around the vehicle. Driving is also done.

  However, even if an ultra-wide-angle camera is used as such a camera outside the vehicle, for example, when a vehicle is put in the garage while the vehicle is moved backwards, and there are obstacles around the rear side of the vehicle, the vehicle is used for backward shooting when the vehicle is moved backward. Being out of the camera's field of view, it becomes impossible to know the state of the obstacle. As a countermeasure, it is also conceivable to provide super wide-angle cameras on both sides of the vehicle so that the portion outside the field of view of the rear photographing camera can be seen by the side photographing camera.

  However, in the image of the ultra-wide-angle camera as described above, the appearance of the object is greatly different between the central part and the peripheral part of the photographed image, and the positional relationship between the object and the vehicle is clear just by looking at the photographed image. Instead, it may cause unexpected collisions and contact accidents. In particular, even if the image from the rear camera is switched to the image from the side camera, the image of the object such as an obstacle displayed on the screen at that time is very different, and the actual state of the object must be grasped. Is not easy.

  Therefore, a plurality of cameras for shooting outside the vehicle are mounted on the vehicle, the images shot by these cameras are combined, and an image is formed as if the surroundings of the vehicle were shot from the camera installed above the vehicle. It has been proposed to be displayed. When using such a plurality of cameras, for example, as shown in FIG. 6, a vehicle right-side shooting camera 42 for shooting the right side of the vehicle 41, a vehicle left-side shooting camera 43 for shooting the left side of the vehicle 41, A total of four cameras, a vehicle rear-facing camera 44 that captures the rear and a vehicle front-facing camera 45 that captures the front side of the vehicle 41, are mounted, and all the images of these cameras are combined. For example, FIG. As schematically shown in c), it has been proposed to display on the monitor an image as if the entire periphery of the vehicle 41 was photographed by a single camera provided directly above the vehicle. This method is also referred to as a top view system, and two or three cameras for photographing the outside of the vehicle are appropriately selected as necessary, and the installation positions are also selected and installed at various positions.

  Note that when displaying a composite image having a virtual viewpoint above the vehicle as described above, changing the position of the virtual viewpoint, the direction of the line of sight, the focal length, and the like according to the traveling state of the vehicle is disclosed in Patent Document 1 is disclosed.

  On the other hand, the range of the area around the vehicle that can be seen while driving is different depending on individual differences such as the driver's seating height, etc. There is a blind spot that can not be. A schematic diagram of an example of such a blind spot is shown in FIG. In FIG. 7, even if the driver of the vehicle has the same seat, the position of the eyes varies depending on the height of the seat. Here, the position of the eyes of the person A1 with a low seat height is at the position of the height H1 from the road surface B, whereas the position of the eyes of the person A2 with a high seat height is at the position of the height H2 from the road surface B. On the other hand, when looking forward in this state as shown in FIG. 6A, in the illustrated example, from the position of each person's eyes, it is in front of the ground positions f1 and f2 when the front end f of the hood of the vehicle is viewed. The side cannot be seen directly. Therefore, at this time, a person with a low seating height has a blind spot ahead of the portion of the distance Sf1 from the eye position. In addition, for a person with a high sitting height, a portion at a distance Sf2 from the position of the eyes is a blind spot ahead.

Further, as shown in FIG. 7B, with respect to the side of the vehicle, from the position of each person's eyes, the front side of the right side window m1 and m2 when looking at the lower end m of the right window is directly eyes. I can't see it. Therefore, at this time, a person with a low sitting height has a right blind spot at a distance Sm1 from the sitting position, and a person with a high sitting height has a right blind spot at a distance Sm2. Similarly, on the left side of the driver, when viewing the lower end h of the left window, distances Sh1 and Sh2 to the ground positions h1 and h2 are left blind spots. In this way, for example, as shown in FIG. 7C, a blind spot area S1 of a person with a low sitting height exists around the center vehicle C, and a narrower blind spot area S2 exists for a person with a high sitting height. I understand that.
JP 2000-613188 A

  As described above, the driver has a blind spot area around the vehicle. When the vehicle is put in a garage, the driver must perform safe driving in consideration of such a blind spot area. The above-mentioned top view system is effective in that case, and the blind spot area around the vehicle can be displayed in an easy-to-see manner by a composite image of a camera that captures the outside of the vehicle.

  However, in this case, as shown in FIG. 1 as a conventional display, in many cases, the top view image is a display range centered on the vehicle, whereas in the blind spot area, the driver's seat is mainly the vehicle as described above. Because the vehicle's right and left sides are different and the front and rear sides are different due to the position being deviated from front to back and left and right from the center of In this case, a large blind spot area is left behind, and the display without considering the blind spot area is performed. This also applies to the technology disclosed in Patent Document 1, and even if it is disclosed that the range of the top view is moved according to the movement state of the vehicle, the above-described problem cannot be solved.

  Therefore, the present invention provides a vehicle blind spot assist system that can move or change the display range in consideration of covering the driver's blind spot area when displaying the surroundings of the vehicle with the top view system. The main purpose is to do.

  In order to solve the above problems, a vehicle blind spot assist system according to the present invention inputs a captured image of a plurality of cameras that capture the periphery of the vehicle and forms a composite image viewed from a virtual viewpoint, and A blind spot area calculation unit that calculates a blind spot area around the vehicle as viewed from the driver, wherein the top view image forming unit is a top view of a range moved to a side that covers the blind spot area calculated by the blind spot area calculation unit An image is formed.

  According to another aspect of the present invention, in the vehicle blind spot assist system, the blind spot area computing unit computes based on the driver's viewpoint position and vehicle data detected by the viewpoint detector. It is characterized by.

  According to another aspect of the present invention, in the vehicle blind spot assist system, when the position of the viewpoint detected by the viewpoint detector does not move for a predetermined time or more in the vehicle blind spot assist system, the blind spot area is calculated based on the driver's viewpoint position. It is characterized by performing.

  According to another aspect of the present invention, in the vehicle blind spot assist system, the blind spot area calculation unit performs a calculation based on data related to a driver's eye position and vehicle data. To

  Further, in the vehicle blind spot assist system according to the present invention, in the vehicle blind spot assist system, a top view image of a range moved to a side covering the blind spot area is a driver seat side corner portion in front of the vehicle in the blind spot area. The top-view image is an image in which the corners on the driver's seat in front of the vehicle coincide with each other.

  Further, in the vehicle blind spot assist system according to the present invention, in the vehicle blind spot assist system, a top view image of a range moved to a side covering the blind spot area is vertical or horizontal direction of the top view image from the blind spot area. When at least one of the two is wide, the image is characterized in that the image uniformly covers both sides of the blind spot area.

  Further, another vehicle blind spot assist system according to the present invention is characterized in that, in the vehicle blind spot assist system, the top view image forming unit forms an image covering all of the blind spot area in accordance with the blind spot area. To do.

  Since the present invention is configured as described above, the display range can be moved or changed in consideration of covering the driver's blind spot area when displaying the surroundings of the vehicle by the top view system. This enables safe operation.

  The present invention aims at covering a driver's blind spot area with an image of a top view system, and inputs a photographed image of a plurality of cameras that photograph the surroundings of the vehicle to form a composite image viewed from a virtual viewpoint. And a blind spot area calculating section that calculates a blind spot area around the vehicle as viewed from the driver, and the top view image forming section has moved to a side that covers the blind spot area calculated in the blind spot area calculating section. Realized by forming a top-view image of the range.

  FIG. 1 shows a functional block diagram of an embodiment of a vehicle blind spot assist system according to the present invention. The vehicle blind spot assisting system 1 shown in FIG. 1 has a top view system similar to that of FIG. 6, and the top view image forming unit 2 captures image data of four external photographing cameras shown in the figure. In the same manner as above, an image around the vehicle having a virtual viewpoint above the vehicle is formed. In that case, as will be described later, an image in a state where the lower part is viewed from an arbitrary viewpoint above the vehicle so that an image of the area designated by the blind spot area cover top view image display range designation unit 12 can be formed. To be able to form.

  In the example shown in FIG. 1, the vehicle data input unit 3 is provided. As basic data for calculating the blind spot region as shown in FIG. 7C, the position of the driver's eyes, the height of the bonnet tip, and the driving The vehicle data such as the distance from the seat, the height of the lower end of the left and right windows and the distance from the driver's seat, the height of the lower end of the rear window and the distance from the driver's seat are input. At the time of this input, it is possible to prepare data of a list in which the manufacturer and format of the vehicle and these data correspond to each other in advance, and read the input by instructing the vehicle format or the like. Further, when this vehicle blind spot assist system is attached at the time of shipment of the vehicle, the data can be input in advance at the factory before shipment. Since this input data does not change unless it is replaced with another vehicle once this input data is input, it is stored in the vehicle data storage unit 4.

  Furthermore, since the driver data input unit 5 is provided and the position of the eyes varies greatly depending on the height and sitting height of the driver, the present invention can be operated even when a device for detecting the driver's viewpoint described later is not provided. As described above, when the driver uses this system, the driver's eye position can be predicted by inputting the driver's sitting height in advance. At this time, since the driver often adjusts the seat position to his / herself by changing the front / rear direction and the vertical direction, the driver's seat data is also input here as data specific to the driver, and the driver's eyes are more accurate. The position may be predicted. Since these data are not changed unless the driver moves the seat or the driver changes, this data is stored in the driver data storage unit 6.

  In the example shown in FIG. 1, the driver's face is photographed so that the driver data input unit 5 can detect the position of the driver's eyes without inputting data related to the individual driver. A viewpoint detection unit 7 is provided for inputting a camera image to detect a viewpoint as an eye position. Therefore, when the viewpoint detection unit 7 is provided, the driver data input unit 5 and the driver data storage unit 6 are not particularly necessary. When detecting the position of the eyes by photographing the driver's face, for example, as shown in FIG. 5, a camera 16 for photographing the face of the driver driving the vehicle 15 is attached and the photographed image is analyzed. The viewpoint 17 that is the position of the driver's eyes can be detected, and such a technique conventionally detects the movement of the eyeball by photographing the driver's face, detects the line of sight, Techniques have been developed for detecting a sleep state, detecting an object being watched by the driver, and the like, and the viewpoint can be easily detected by using a part of this technique.

  In the viewpoint movement time calculation unit 8, the head is temporarily moved as if the driver temporarily changed his / her posture by, for example, an audio operation while the driver's viewpoint is constantly detected. When the viewpoint moves, the time is calculated when the viewpoint moves so that the processing of changing the calculation base point of the blind spot range described later is not performed. The viewpoint position calculation unit 9 calculates the position of the viewpoint in the vehicle based on the detection value of the viewpoint detection unit 7 when the viewpoint has not changed significantly for a predetermined time such as 10 seconds or more as described above.

  In the blind spot area calculation unit 10, when the viewpoint detection unit 5 is not provided, the data stored in the vehicle data storage unit 4 and the data stored in the driver data storage unit 6 are read, and the blind spot area is shown in FIG. It calculates by the technique like this. In addition, when the viewpoint detection unit 7 is provided, the driving posture of the driver is not affected by the short-term viewpoint change by the calculation result of the viewpoint position calculation unit 9 and the data of the vehicle data storage unit 4. A blind spot area corresponding to the position of the viewpoint corresponding to the position change of the front, back, left and right of the head due to the change of the head is calculated.

  In the example shown in FIG. 1, a blind spot area reference point calculation unit 11 is provided, and in the blind spot area shown as a conventional display in FIG. 1, the corner at the upper right in the figure, which is the corner on the driver's seat in front of the vehicle, is set as the reference point Ps. The position of the reference point Ps with respect to the vehicle is calculated. Based on this calculation result, the blind spot area cover top view image display range specifying unit 12 calculates a range for displaying the top view image, and instructs the top view image forming unit 2 to display it. Accordingly, when the same area as the conventional top view image is set as the image display range, the top view image is displayed in a state where the driver side corner Pt in front of the vehicle of the top view image is positioned at the blind spot reference point Ps. Will be.

  The range of the top view image displayed at this time can be set to the same area as the conventional display range as described above, or an image that covers the entire blind spot area calculated by the blind spot area calculation unit 10 can be formed. This also makes it possible to eliminate blind spots at all in this system. Further, as described later, in addition to displaying an image in which the corner Pt of the top view image coincides with the blind spot reference point Ps, an image shifted as necessary may be displayed. When the display range is arbitrarily set in this way, a virtual viewpoint is arranged above the range according to the display range, and a predetermined image is synthesized.

  In the embodiment of the present invention comprising the functional blocks as described above, for example, it can be carried out by sequentially operating according to the operation flow shown in FIG. In the blind spot area cover top view image forming process shown in FIG. 2, basic vehicle data for first blind spot area setting is input (step S1). This processing is performed by inputting data necessary for calculation of the blind spot area peculiar to each vehicle in the vehicle data input unit 3 in FIG. 1, and in this case, the vehicle manufacturer and model are instructed in advance as described above. Can be input.

  Next, an image forming area for forming a steady top view image is set. For example, as shown in the steady top view image forming area in FIG. 2B, this top view is performed when a conventional image with a viewpoint existing above the substantially central portion of the vehicle is synthesized. The system normally sets which range is the image display range, and the image display range normally does not change like the area indicated by T in FIG. 2B, for example. You can leave it. However, as described in Patent Document 1, when the display range changes due to the movement of the vehicle, the actual top view image forming area may be set based on the change data of the display range due to the movement of the vehicle at that time. This data can be set by using data provided as data for forming a top view image in the top view image forming unit 2 of FIG.

  Thereafter, the viewpoint is detected (step S3). This operation is performed in the viewpoint detection unit 7 in FIG. Next, the height and front / rear position of the detected viewpoint, and the left / right positions are measured (step S4). Here, the height of the viewpoint is mainly important for the calculation of the blind spot area later, and it may be necessary to measure the front-rear position according to the change of the driving posture, and further, drive closer to the window side during driving If the left and right positions are measured in consideration of the situation, the viewpoint position can be measured more accurately.

  Next, it is determined whether or not the viewpoint is within a predetermined range for a predetermined time (step S5). This operation is performed when the viewpoint moving time calculation unit 8 in FIG. 1 calculates the time during which the viewpoint exists within a predetermined range for a certain time or more, and when the viewpoint position calculation unit detects that a certain time or more has elapsed from the data, This is performed by outputting the viewpoint position detected by the detection unit 7 to the blind spot area calculation unit 10. In step S5, when the viewpoint is not within the predetermined range for a predetermined time or more, that is, when the viewpoint is temporarily moved, the process returns to step S2 and the above operation is repeated.

  If it is determined in step S5 that the viewpoint is within the predetermined range for a predetermined time or longer, then the blind spot area is calculated (step S6). This is performed in the blind spot area calculation unit 10 of FIG. 1, thereby setting the blind spot area S of FIG. Next, the driver's side corner position in front of the vehicle in the blind spot area as the blind spot area reference point is calculated (step S7). This position corresponds to the corner portion Ps of the blind spot region S in FIG.

  At the end of these processes, the top view image area is set so that the driver's seat side front end corner (Pt) of the top view image formation area matches the driver's seat side front end corner (Ps) of the blind spot area, The composite display of the top view image in that range is performed. This state is shown in FIG. 2C as a top view image T in a state where the two corners Pt and Ps coincide with each other, thereby displaying a portion that is not a blind spot area in the conventional top view image, and a large blind spot. Even when the area is left, the display portion of the area that is not a meaningless blind area according to the present invention can be used as the display area of the blind area, thereby enabling safer driving of the vehicle.

  In the above example, the case where the blind spot area is calculated based on an accurate viewpoint with the viewpoint detection means and the top view image is larger in both the vertical and horizontal directions than the blind spot area has been described, but the viewpoint detection that is expensive as equipment FIG. 3 shows a top view display mode in which a viewpoint is simply set without using any means, and the top view image is taken into consideration when the top view image is larger or smaller than the blind spot area, or both are larger.

  In the example of the blind spot area cover top view image forming process shown in FIG. 3A, first, the basic vehicle data for setting the blind spot area is input as in the example of FIG. 2 (step S11). Thereafter, in the illustrated example, blind spot area setting driver data is input (step S12). This process is performed by inputting various data relating to the viewpoint such as the height of the driver, including the forward / backward movement of the driver's seat and the detection of vertical movement, in the driver data input unit 5 of FIG.

  Next, as in step S6 of FIG. 2, the blind spot area is calculated (step S13), and then it is determined whether or not the blind spot area is wider in the horizontal direction than the top view range (step S14). That is, as shown in FIG. 3B, it is determined whether or not the lateral length Sw of the blind spot area S is longer than the lateral length Tw of the top view range T. The blind spot area S is wider than the top view range T in the lateral direction, that is, it is determined that Sw> Tw. At that time, in step S15, it is determined whether the blind spot area S is wider than the top view range T in the vertical direction. That is, as shown in FIG. 4B, it is determined whether or not the vertical length SL of the blind spot area S is longer than the vertical length TL of the top view range T. That is, when Sw> Tw and SL> TL, as in step S8 of FIG. 2, the driver seat side front end portion of the top view image forming area and the driver seat side front end corner portion of the blind spot area coincide with each other. The top view image is displayed (step S16). This display mode is a mode of display as shown in (a2) under the conditions shown in FIG. 4 (a1).

  In step S15, it is determined whether or not the blind spot area is wider in the vertical direction than the top view range. When it is determined that the blind spot area is wider in the vertical direction, Sw> Tw and SL ≦ TL, and FIG. In this case, in the processing example shown in FIG. 2, the display is as shown in (b2), whereas in the example of FIG. 3, as shown in FIG. The direction (vertical direction in the figure) is displayed wider by the same length than the longitudinal direction of the blind spot area. As a result, before and after the blind spot area, the peripheral part of the blind spot area, which is not necessarily a strict area, can be covered evenly using the margin of the top view display, and the blind spot area can be covered more appropriately. Can do.

  If it is determined in step S14 that the blind spot area is not wider in the lateral direction than the top view range, it is determined in step S18 whether the blind spot area is wider in the lateral direction than the top view range as in step S15. Determine. As a result, when it is determined that it is wide, Sw ≦ Tw and when SL> TL, and in this case, the state shown in FIG. 4 (c1), the display example of FIG. On the other hand, in this example, as shown in (c3), the image is displayed uniformly and widely in the left-right direction of the blind spot area. The purpose of this display method is the same as the display of (b3).

  When it is determined in step S18 that the blind spot area is not wider in the horizontal direction than the top view range, it is Sw ≦ Tw and SL> TL. In this case, the state is as shown in FIG. In the display example of FIG. 2, the display is as shown in (d2), whereas in this example, the display is equally wide in front, rear, left, and right of the blind spot area as shown in (d3).

  The auxiliary display method of the blind spot area by the top view display as described above may be always displayed in this manner in a vehicle equipped with the top view system, but otherwise, the conventional display method is a normal display, Such display may be performed when the user instructs. In addition, since the display range of the top view can be arbitrarily set in many cases, the display corresponding to the blind spot area may be performed as described above, and in this case, the top view display range as described above may be used. There is no need to perform various movement processes.

It is a functional block diagram of the Example of this invention. It is an operation | movement flowchart of the Example. It is another operation | movement flowchart of the Example. It is a figure which shows the various display aspects of the top view image in the Example of this invention. It is a figure which shows a viewpoint detection aspect. It is explanatory drawing of the conventional top view system. It is a figure which shows the state of the blind spot around the vehicle at the time of a driving | operation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Vehicle blind spot assistance system 2 Top view image formation part 3 Vehicle data input part 4 Vehicle data storage part 5 Driver data input part 6 Driver data storage part 7 View point detection part 8 View point movement time calculation part 9 View point position calculation part 10 Blind spot Area calculation section 11 Blind spot area reference point calculation section 12 Blind spot area cover top view image display range designation section

Claims (7)

A top-view image forming unit that forms a composite image viewed from a virtual viewpoint by inputting captured images of a plurality of cameras that capture the periphery of the vehicle;
A blind spot area calculation unit that calculates a blind spot area around the vehicle as viewed from the driver,
The top-view image forming unit forms a top-view image of a range moved to a side that covers the blind spot area calculated by the blind spot area calculation unit.   The vehicle blind spot assisting system according to claim 1, wherein the blind spot area calculation unit performs calculation based on the position of the driver's viewpoint detected by the viewpoint detection unit and vehicle data.   The vehicle blind spot assist system according to claim 2, wherein when the position of the viewpoint detected by the viewpoint detection unit does not move for a predetermined time or longer, a blind spot area is calculated based on the position of the driver's viewpoint.   3. The vehicle blind spot assisting system according to claim 2, wherein the blind spot area computing unit performs computation based on data related to a driver's eye position and vehicle data.   The top view image of the range moved to the side that covers the blind spot area is an image in which the driver's seat side corner in front of the vehicle in the blind spot area matches the driver seat side corner in front of the vehicle in the top view image. The vehicle blind spot assist system according to claim 1.   The top view image of the range moved to the side that covers the blind spot area is an image that equally covers both sides of the blind spot area when the top view image is wider in at least one of the vertical direction and the horizontal direction than the blind spot area. The vehicle blind spot assist system according to claim 1.   2. The vehicle blind spot assisting system according to claim 1, wherein the top-view image forming unit forms an image that covers all of the blind spot area in accordance with the blind spot area.

Images