JP2009005054A - Driving support device, driving support method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driving support device, a driving support method, and a program that enable to display an image, which prevents both a driver and a fellow passenger from feeling a sense of incongruity, on a display device provided in a front pillar. <P>SOLUTION: In the driving support device, when a head position 51D of a driver 51 is in a first region (S11-S21:YES), an image of each blind spot region A1, A2, generated by a pillar P, of the driver 51 and an occupant 52 on a front passenger seat G is displayed on a multi-view display 4 mounted inside the pillar P as a video image viewed from each direction of the driver and the occupant (S22). When the head position 51D of the driver 51 is in a second region (S21:NO), image data of a projection part CP1 partitioned by the blind spot region A1 of the driver 51 are displayed as video-image data for the driver's seat F side and video-image data for the front passenger seat G side of the multi-view display 4 (S23). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a driving support device, a driving support method, and a program that enable a blind spot blocked by a front pillar of a vehicle to be visually recognized.

Conventionally, various driving support devices that make it possible to visually recognize a blind spot blocked by a front pillar of a vehicle have been proposed.
For example, there is a driving assistance device that captures a blind spot area caused by a front pillar of a vehicle with an imaging device that captures the outside of the vehicle and displays this captured image on a display device provided inside the front pillar (for example, Patent Document 1). reference.).
JP 2005-204132 A (paragraphs (0013) to (0032), FIGS. 1 to 13)

  However, in the driving assistance apparatus described in Patent Document 1 described above, image data of a blind spot area generated by the driver's front pillar in accordance with the driver's viewpoint is displayed on the display device. For passengers (for example, passengers in the passenger seat), since the scenery is not continuously displayed, there is a problem in that the scenery that is actually seen deviates from the scenery that is displayed on the display device.

  Therefore, the present invention has been made to solve the above-described problems, and an image is provided on the front pillar where the driver and the passenger (for example, a passenger in the passenger seat) do not feel discomfort. It is an object to provide a driving support device, a driving support method, and a program that can be displayed on a displayed display device.

  In order to achieve the above object, the driving support apparatus according to claim 1 obtains image data including a blind spot area of a driver generated by a front pillar (P) of the vehicle from a photographing unit (6) for photographing the outside of the vehicle. A data acquisition means (22), a driver side extraction means (20) for extracting a part of the image data defined by the driver's blind spot area from the image data, and a compartment by the passenger's blind spot area from the image data. Passenger side extraction means (20) for extracting image data of a portion to be processed, and provided at the inner side of the front pillar, it is possible to display individual images at least when viewed from each direction of the driver and the passenger The multi-view display means (4) displays the image data extracted by the driver side extraction means as an image of the direction seen from the driver, A multi-view display control means for displaying the image data extracted by the passenger's side extraction means as the direction of the image as viewed from the passenger (20), characterized by comprising a.

  The driving support apparatus according to claim 2 is the driving support apparatus (1) according to claim 1, wherein the multi-view display control means (20) is configured such that the head position of the driver is the multi-view display means. Head position determination means (20) for determining whether or not the image viewed from the passenger's direction is visible, and the multi-view display control means (20) includes the driver's head When it is determined that the position is at a position where the video viewed from the direction of the passenger on the multi-view display means can be seen, the image data extracted by the driver-side extraction means (20) is taken with the driver. It controls to display as an image of each direction seen from the person.

  According to a third aspect of the present invention, there is provided a driving support apparatus according to the first or second aspect, wherein the image data that sequentially stores the image data acquired by the image data acquisition means (22) is provided. Storage means (11), and extraction determination means (20) for determining whether or not image data of a portion partitioned by the blind spot area of the fellow passenger can be extracted from the image data acquired by the image data acquisition means (22). And the passenger side extraction means (20) is unable to extract the image data of the portion partitioned by the blind spot area of the passenger from the image data acquired by the image data acquisition means (22). If it is determined, the image data of the portion partitioned by the blind spot area of the passenger is extracted from the image data stored at a position a predetermined distance before the own vehicle position. Characterized in that it.

  A driving support apparatus according to claim 4 is the driving support apparatus (1) according to any one of claims 1 to 3, wherein the image data acquired by the image data acquisition means (22) The driver-side virtual plane setting means (20) for setting the driver-side virtual plane for projecting the image data of the driver's blind spot area and the photographing surface of the photographing means (6) are defined by the driver's blind spot area. An imaging surface setting means (20) for setting the position passing through the end point of the divided driver-side virtual plane, and an image of the blind spot area of the passenger among the image data acquired by the image data acquisition means (22) Passenger-side virtual plane setting means (20) for setting a passenger-side virtual plane for projecting data, and the image data by focusing the photographing means on the photographing surface set by the photographing surface setting means. Focus control means (20, 22) for controlling to obtain, driver-side image processing means (20) for converting the image data extracted by the driver-side extraction means into the driver-side virtual plane, and Passenger-side image processing means (20) for converting the image data extracted by the passenger-side extraction means into the passenger-side virtual plane, and the multi-view display control means (20) includes the driver The image data coordinate-converted to the driver-side virtual plane by the side image processing means is displayed as an image in the direction seen from the driver's seat, and coordinate-converted to the passenger-side virtual plane by the passenger-side image processing means The image data thus displayed is displayed as an image in a direction seen from the passenger.

  According to a fifth aspect of the present invention, there is provided the driving support device according to the fourth aspect, wherein the photographing surface setting means (20) sets the photographing surface of the photographing means from the driver-side virtual plane. Is set at a position close to the vehicle.

  The driving support device according to claim 6 is the driving support device (1) according to claim 5, wherein the passenger-side virtual plane setting means (20) sets the passenger-side virtual plane to the photographing means. The position is set farther from the vehicle than the photographing surface.

  The driving support device according to claim 7 is the driving support device (1) according to any one of claims 4 to 6, wherein the driver-side virtual plane setting means (20) A position passing through the driver-side reference position after setting the driver-side reference position on the road surface in the blind spot area of the driver based on the head position and the projection position on the road surface of the front pillar. The driver side virtual plane is set, and the imaging plane setting means (20) is configured such that the direction of the optical axis of the imaging means, the projected position of the driver's head position on the road surface, and the driver side Based on the azimuth of a straight line connecting the reference positions, an end point of the driver-side virtual plane where the imaging plane intersects is selected.

  The driving support device according to claim 8 is the driving support device according to claim 7, wherein the passenger-side virtual plane setting means (20) includes the head position of the passenger and the front pillar. And the passenger side reference position on the road surface in the blind spot area of the passenger based on the projected position on each road surface, and then the head position of the passenger and the passenger side reference position are connected. The passenger-side virtual plane is set at a position passing through an end point far from the vehicle among both end points that are orthogonal to a straight line and the imaging plane intersects the blind spot area of the passenger. .

  Furthermore, the driving support apparatus according to claim 9 is the driving support apparatus (1) according to any one of claims 4 to 6, wherein the image data storage means (11) stores the image data of the photographing surface. Whether the extraction determination means (20) can sequentially extract the image data of the imaging surface of the portion partitioned by the blind spot area of the passenger from the imaging surface image data set by the imaging surface setting means. The passenger side extraction means (20) determines whether or not the passenger side extraction means (20) captures image data of a part of the photographing surface divided by the blind spot area of the fellow passenger from the image data of the photographing surface set by the photographing surface setting means. Is determined to be unable to be extracted, the image data of the photographing surface of the portion partitioned by the blind spot area of the passenger from the image data of the photographing surface stored at a position a predetermined distance before the own vehicle position. Extracting, the passenger-side image processing means (20), characterized in that the coordinate converting the image data extracted by the passenger's side extraction unit to the passenger's virtual plane.

  The driving support method according to claim 10 is an image data acquisition step (S11 to S17) for acquiring image data including a driver's blind spot area generated by a front pillar of the vehicle from an imaging unit that images the outside of the vehicle. A driver-side extraction step (S18) for extracting image data of a portion partitioned by the driver's blind spot area from the image data, and extracting image data of a portion partitioned by the passenger's blind spot area from the image data. The passenger side extraction step (S19), and multi-view display means provided inside the front pillar and capable of displaying individual images at least when viewed from each direction of the driver and the passenger, The image data extracted in the driver side extraction step is displayed as a video in the direction seen from the driver and extracted in the passenger side extraction step. A multi-view display control step (S20 to S23) for controlling the image data to be displayed as the direction of the image as viewed from the passenger, characterized by comprising a.

  Furthermore, the program according to claim 11 is an image data acquisition step (S11 to S17) for acquiring image data including a driver's blind spot area generated by a front pillar of the vehicle from a photographing means for photographing the outside of the vehicle. A driver-side extraction step (S18) for extracting a portion of the image data defined by the driver's blind spot area from the image data; and a portion of the image data defined by the passenger's blind spot region from the image data. Passenger side extraction step (S19) to extract, and multi-view display means provided inside the front pillar and capable of displaying individual images when viewed from each direction of at least the driver and the passenger, The image data extracted in the driver side extraction step is displayed as an image of the direction seen from the driver, and the passenger side A multi-view display control step (S20 to S23) for controlling so that output image data extracted in step displays as a direction of the video as seen from the passenger, a program for execution.

  In the driving support apparatus according to claim 1 having the above-described configuration, the image of each blind spot area of the driver and the passenger is displayed on the multi-view display means provided inside the front pillar as an image viewed from each direction. It becomes possible. For this reason, it becomes possible for the driver and the passenger to display an image of the blind spot area in which both the driver and passenger do not feel uncomfortable on the multi-view display means provided on the front pillar.

  Further, in the driving support device according to claim 2, when the head position of the driver is at a position where the image viewed from the passenger's direction of the multi-view display means can be seen, the driving assistance device is partitioned by the driver's blind spot area. The image data of the portion is displayed as an image in each direction viewed from the driver and the passenger of the multi-view display means. Therefore, even when the driver's head position is at a position where the image viewed from the passenger's direction of the multi-view display means can be seen, the image of the blind spot area where the driver and the passenger do not feel discomfort is displayed on the front pillar. It is possible to display on the multi-view display means provided in.

  In the driving support device according to claim 3, when it is not possible to extract the image data of the portion partitioned by the blind spot area of the passenger from the image data acquired by the image data acquisition means, a predetermined distance before the own vehicle position. The image data of the portion partitioned by the blind spot area of the passenger is extracted from the image data stored at the position. For this reason, even when the angle of view of the photographing means is small, the image data of the portion defined by the blind spot area of the fellow passenger can be extracted and displayed as an image in the direction seen from the fellow passenger of the multi-view display means. It becomes possible. Therefore, even when the angle of view of the photographing means is small, it is possible for the driver and the passenger to display an image on which the driver does not feel uncomfortable on the multi-view display means provided on the front pillar.

  Further, in the driving support apparatus according to claim 4, in order to set the imaging surface of the imaging means to a position passing through the end point of the driver-side virtual plane defined by the driver's blind spot area, the driver of the multi-view display means The video in the direction viewed from the side is highly continuous, and a clear and natural image can be displayed. In addition, since the image data of the photographing surface of the portion divided by the blind spot area of the passenger is extracted from the image data of the photographing surface and displayed as a video in the direction viewed from the fellow passenger of the multi-view display means, The video in the direction viewed from the passenger of the view display means is also highly continuous and can display a clear and natural image. Therefore, the driver and the passenger can both display an image that does not feel uncomfortable on the multi-view display means provided on the front pillar.

  In the driving support device according to the fifth aspect, the photographing surface of the photographing device is set at a position closer to the vehicle than the driver-side virtual plane, and thus the focus is on the surplus area of the image on the photographing surface photographed by the photographing means. The images in the direction viewed from the driver of the multi-view display means are more continuous and can display a clear and natural image.

  Further, in the driving support device according to the sixth aspect, the passenger-side virtual plane is set at a position farther from the vehicle than the imaging surface of the imaging device, so that it is focused on the surplus area of the image on the imaging surface captured by the imaging means. The images in the direction seen from the passenger of the multi-view display means are more continuous, and a clear and natural image can be displayed.

  Further, in the driving support device according to claim 7, according to the azimuth of the optical axis of the photographing means and the azimuth of the straight line connecting the projection position on the road surface of the driver's head position and the driver side reference position, The end point through which the image plane passes is changed. For this reason, it becomes possible to always set the imaging surface in front of the driver-side virtual plane.

  In the driving support device according to the eighth aspect, the vehicle is out of the end points that are orthogonal to a straight line connecting the passenger's head position and the passenger-side reference position and whose imaging plane intersects the blind spot area of the passenger. The passenger-side virtual plane is set at a position passing through the end point at a position far from the passenger. For this reason, it becomes possible to always set the imaging surface in front of the passenger-side virtual plane.

  Furthermore, in the driving support device according to claim 9, when the image data of the photographing surface of the portion partitioned by the blind spot area of the passenger cannot be extracted from the image data of the photographing surface, the position a predetermined distance before the own vehicle position. The image data of the photographing surface of the portion partitioned by the blind spot area of the passenger is extracted from the image data of the photographing surface stored in step (b). For this reason, even when the angle of view of the photographing means is small, the image data of the photographing surface of the portion defined by the blind spot area of the fellow passenger is extracted and displayed as an image in the direction viewed from the fellow passenger of the multi-view display means. It is possible to display a clear and natural image with high continuity even in the video viewed from the passenger of the multi-view display means. Therefore, even when the angle of view of the photographing means is small, it is possible for the driver and the passenger to display an image on which the driver does not feel uncomfortable on the multi-view display means provided on the front pillar.

  In the driving support method according to the tenth aspect, the image of each blind spot area of the driver and the passenger can be displayed on the multi-view display means provided inside the front pillar as an image viewed from each direction. It becomes possible. For this reason, it becomes possible for the driver and the passenger to display an image of the blind spot area in which both the driver and passenger do not feel uncomfortable on the multi-view display means provided on the front pillar.

  Furthermore, in the program according to claim 11, when the computer reads the program, the computer displays images of the blind spot areas of the driver and the passenger on the multi-view display means provided inside the front pillar, respectively. It is possible to display it as an image viewed from the direction. For this reason, it becomes possible for the driver and the passenger to display an image of the blind spot area in which both the driver and passenger do not feel uncomfortable on the multi-view display means provided on the front pillar.

  Hereinafter, a driving support device, a driving support method, and a program according to the present invention will be described in detail with reference to the drawings based on a first embodiment and a second embodiment in which the driving support system is embodied.

First, a schematic configuration of the driving support system according to the first embodiment will be described with reference to FIGS. 1 to 4.
FIG. 1 is a block diagram illustrating the configuration of the driving support system 1 according to the first embodiment. FIG. 2 is an explanatory diagram of a position detection sensor that detects the driver's head and the passenger's head in the passenger seat. FIG. 3 is an explanatory diagram of camera mounting positions. FIG. 4 is an explanatory diagram of attachment positions of the camera and the multi-view display.

  The driving support system 1 is mounted on a vehicle C (see FIG. 3), and as shown in FIG. 1, a driving support unit 2 as a driving support device, a display 3, a multi-view display 4, a speaker 5, and a camera 6 as an imaging device. Two sets of first to third position detection sensors 8A to 8C (see FIG. 2) are provided.

  The driving support unit 2 includes a control unit 10, a nonvolatile main memory 11, a ROM 12, and a GPS receiving unit 13. The control unit 10 is a CPU, MPU, ASIC, or the like, and performs main control of each process according to the driving support program stored in the ROM 12. The main memory 11 temporarily stores the calculation result of the control unit 10.

  Moreover, the control part 10 acquires the satellite orbit information and time information which the GPS receiving part 13 received from the GPS satellite, and calculates the absolute position of the own vehicle by radio wave navigation. Further, the control unit 10 receives vehicle speed pulses and angular velocities from the vehicle speed sensor 30 and the gyro 31 provided in the vehicle C via the vehicle side I / F unit 14 of the driving support unit 2. And the control part 10 calculates the relative position from a reference position by the autonomous navigation using a vehicle speed pulse and an angular velocity, and pinpoints the own vehicle position combining with the absolute position calculated by the radio wave navigation.

  The driving support unit 2 includes a geographic data storage unit 15. The geographic data storage unit 15 is an external storage medium such as an internal hard disk or an optical disk. In the geographic data storage unit 15, each route network data (hereinafter referred to as route data 16) as map data for searching for a route to the destination, and map data for outputting the map screen 3A to the display 3 are displayed. Each map drawing data 17 is stored.

  The route data 16 is data relating to roads in the mesh dividing the whole country. The route data 16 includes data such as identifiers of each mesh, node data related to nodes indicating intersections and road endpoints, identifiers of links connecting the nodes, and link costs. The control unit 10 uses the route data 16 to search for a recommended route to the destination, and determines whether the vehicle C is approaching a guide point such as an intersection.

  The map drawing data 17 is data for drawing a road shape, a background, and the like, and is stored for each mesh obtained by dividing a map of the whole country. The map drawing data 17 stores data on road markings such as a center line, a white line that divides a roadside zone, a zebra zone, and a pedestrian crossing, and road surface installations such as traffic lights. Specifically, the types of road markings, the position coordinates of road markings, the types of road surface installation objects, the position coordinates of road surface installation objects, and the like are stored in association with the intersections and curves of each point.

  As shown in FIG. 1, the driving support unit 2 includes a map drawing processor 18. The map drawing processor 18 reads out the map drawing data 17 for drawing a map around the vehicle position from the geographic data storage unit 15, generates map output data, and displays the map screen 3A based on the map output data. Displayed on the display 3. Further, the map drawing processor 18 superimposes a vehicle position mark 3B indicating the vehicle position on the map screen 3A.

  In addition, the driving support unit 2 includes an audio processor 24. The audio processor 24 has an audio file (not shown), and outputs, for example, audio from the speaker 5 that guides the route to the destination. Further, the driving support unit 2 includes an external input I / F unit 25. The external input I / F unit 25 receives an input signal based on a user input operation from the operation switch 26 adjacent to the display 3 and the touch panel display 3 and outputs the input signal to the control unit 10.

  The driving support unit 2 includes a sensor I / F unit 23 that constitutes a detection unit. The sensor I / F unit 23 receives detection signals from the two sets of first to third position detection sensors 8A to 8C (see FIG. 2). The two sets of first to third position detection sensors 8A to 8C are constituted by ultrasonic sensors, and as shown in FIG. It is attached around the occupant 52 (passenger) seated in the seat G.

  Further, the first to third position detection sensors 8A to 8C attached around the driver's seat F respectively detect the front and left roofs R (see FIG. 3) of the driver's seat F in order to detect the head position 51D of the driver 51. (See) and the inner ceiling of the roof R on the upper left side of the driver's seat F. Further, the first to third position detection sensors 8A to 8C attached to the periphery of the passenger seat G respectively detect the front and right roofs R of the passenger seat G in order to detect the head position 52D of the passenger 52 of the passenger seat G. Are arranged on the inner ceiling portion of the roof R on the upper left side of the passenger seat G.

The ultrasonic waves transmitted from the sensor heads (not shown) of the position detection sensors 8A to 8C arranged around the driver 51 are reflected by the head 51D of the driver 51. In addition, ultrasonic waves transmitted from sensor heads (not shown) of the position detection sensors 8A to 8C arranged around the passenger 52 in the passenger seat G are reflected by the head 52D of the passenger 52. And each position detection sensor 8A-8C measures the time after transmitting an ultrasonic wave until receiving a reflected wave, and each relative distance to each head 51D and 52D based on the measurement time, respectively calculate. The calculated relative distances are output to the control unit 10 via the sensor I / F unit 23.
The sensor I / F unit 23 may calculate the relative distances to the heads 51D and 52D based on signals from the position detection sensors 8A to 8C.

  In addition, the control unit 10 is in a state where the head 51 </ b> D of the standard-type driver 51 can move in the state where the driver 10 is seated on the driver's seat F, and the first movement units arranged around the driver 51. Based on the relative distances detected by the third position detection sensors 8A to 8C, the head center position 51DC as the head position is obtained by a known method using triangulation or the like. In addition, the control unit 10 sits on the passenger seat G around the occupant 52 in the passenger seat G and the head movement range in which the head 52D of the standard occupant 52 (passenger) can move. Based on the relative distances detected by the arranged position detection sensors 8A to 8C, the head center position 52DC as the head position is obtained by a known method using triangulation or the like.

  As shown in FIG. 1, the driving support unit 2 includes a video data input unit 22 and an image processor 20. The video data input unit 22 drives the camera 6 provided in the vehicle C based on the control of the control unit 10, captures the image data IM captured by the camera 6, and outputs it to the image processor 20.

  The camera 6 is a camera that captures a color image, and includes an optical mechanism including a lens, a mirror, and the like, a CCD image sensor (none of which is shown), an autofocus mechanism, and the like. As shown in FIGS. 3 and 4, in the first embodiment, the camera 6 has a light beam in the upper right corner as viewed from the passenger compartment of the front window W1 of the vehicle C in accordance with the driver's seat F disposed on the right side of the vehicle. The shaft is attached to the front of the vehicle C. In addition, the camera 6 captures an image of the background of the imaging region Z1 including the front right side of the vehicle C and a part of the right side of the vehicle C.

  Further, the image processor 20 of the driving support unit 2 acquires the image data IM from the camera 6 via the video data input unit 22. In addition, the image processor 20 trims a region blocked by the pillar P in the acquired image data IM and performs image processing for eliminating image distortion. Moreover, the image processor 20 displays the image | video of each blind spot area | region of the driver | operator 51 and the passenger | crew 52 of the front passenger seat G on the multi view display 4 so that it may mention later (refer FIG. 5).

  As shown in FIG. 4, the multi-view display 4 is attached to an inner side surface PA of a front pillar P (hereinafter simply referred to as “pillar P”), and displays individual images when viewed from different directions. This is a known display device that can be used (for example, see Japanese Patent Application Laid-Open No. 2007-8235). Therefore, as shown in FIG. 2, in the coordinate system with the center of the multi-view display 4 as the origin, an image for the driver's seat F side (first region) and an image for the passenger seat G side (second region) are displayed. Each can be displayed individually.

  As a result, as will be described later, an image of a portion that becomes a blind spot area by the pillar P of the driver 51 is displayed as an image for the driver's seat F side of the multi-view display 4. It is possible to see an image in which W2 continues. In addition, an image of a part that becomes a blind spot area by the pillar P of the passenger 52 of the passenger seat G is displayed as an image for the passenger seat G side of the multi-display 4, and the passenger 52 has an image in which the front window W1 and the door window W2 are continuous. Can be seen.

  Next, display processing for displaying images of each blind spot area by the pillar P of the driver 51 and the passenger 52 in the passenger seat G on the multi-view display 4 by the driving support unit 2 configured as described above will be described with reference to FIGS. Based on

  FIG. 5 is a flowchart illustrating a display process in which the driving support unit 2 displays images of each blind spot area by the pillar P of the driver 51 and the passenger 52 in the passenger seat G on the multi-view display 4. FIG. 6 is an explanatory diagram of the driver side reference point PC and the passenger side reference point PS of the passenger seat. FIG. 7 is an explanatory diagram of a driver-side virtual plane and a photographing surface on the driver seat F side. FIG. 8 is an explanatory diagram of the passenger-side virtual plane on the passenger seat G side and the imaging plane. FIG. 9 is an explanatory diagram of the driver-side virtual plane and the photographing surface on the driver seat F side. FIG. 10 is an explanatory diagram of the passenger-side virtual plane on the passenger seat G side and the imaging plane.

As shown in FIG. 5, first, in step (hereinafter abbreviated as “S”) 11, the control unit 10 starts photographing with the camera 6 via the video data input unit 22. Thereby, the image data IM photographed by the camera 6 is sequentially output to the image processor 20.
In S12, the control unit 10 determines the center of the head of the driver 51 from the relative distances to the head 51D of the driver 51 input from the position detection sensors 8A to 8C arranged around the driver 51. The position 51DC is calculated and stored in the main memory 11.

  Subsequently, in S <b> 13, the image processor 20 reads the head center position 51 </ b> DC of the driver 51 from the main memory 11. Then, as shown in FIG. 6, the image processor 20 projects the projection position 61 of the head center position 51DC of the driver 51 onto the road surface J and the center position 62 of projection onto the road surface J of the lower end portion of the pillar P. Is obtained on the basis of the map drawing data 17 at a predetermined distance (approximately 4 m in the first embodiment) from the projection position 61 in the direction of the pillar P toward the pillar P side. Stored in the main memory 11 as a PC.

  Then, the image processor 20 determines the position of the driver-side virtual plane VP based on the coordinates of the driver-side reference point PC and the head center position 51DC of the driver 51. The driver-side virtual plane VP is a plane on which an image taken by the camera 6 is projected. The object on the driver-side virtual plane VP is displayed without being shifted or tilted when the image data IM photographed by the camera 6 is coordinate-converted on the driver-side virtual plane VP. There is.

  As shown in FIG. 7, the image processor 20 includes the driver-side virtual plane VP on a center line LA that includes the driver-side reference point PC and connects the projection position 61 and the driver-side reference point PC. Set it to a position that is perpendicular to it. Further, the image processor 20 divides the driver-side virtual plane VP into a blind spot area of the driver 51 defined by the tangent lines L1 and L2 passing through the head center position 51DC of the driver 51 and the end points P1 and P2 of the pillar P. Set inside A1.

  Subsequently, in S14, the image processor 20 determines the imaging plane CP according to the position of the driver-side virtual plane VP. The imaging plane CP indicates the focal position of the camera 6 and is represented by a plane perpendicular to the optical axis AX of the camera 6. As shown in FIG. 7, the image processor 20 drives the imaging plane CP at a position closer to the vehicle C than the driver-side virtual plane VP and is partitioned by the blind spot area A1 of the driver 51. It is set to a position (predetermined position) that intersects at the left end point VP1 or the right end point VP2 of the person-side virtual plane VP. Here, the imaging plane CP is set at a position close to the head center position 51DC with reference to the head center position 51DC of the driver 51. Note that the end points VPl and VP2 are intersections between the tangent lines L1 and L2 that connect the head center position 51DC and the end points P1 and P2 of the pillar P, respectively, and the driver-side virtual plane VP.

  The method of setting the imaging plane CP will be described in detail. First, the image processor 20 has a center line connecting the orientation of the optical axis AX of the camera 6, the head center position 51DC of the driver 51 and the driver side reference point PC. Compare the orientation of LA. That is, an angle θ1 formed by the optical axis AX and the horizontal direction (Y arrow direction) and an angle θ2 formed by the center line LA and the horizontal direction (Y arrow direction) are calculated. Then, the magnitude of the angle θ1 of the optical axis AX is compared with the magnitude of the angle θ2 of the center line LA. The X arrow direction is a direction parallel to the longitudinal direction of the vehicle C, and the Y arrow direction is a direction orthogonal to the X arrow direction and is a vehicle width direction.

  As shown in FIG. 7, when the angle θ1 of the optical axis AX is smaller than the angle θ2 of the center line LA (θ1 <θ2), the image processor 20 sets the photographing surface CP to the driver-side virtual plane VP. It is set to a position passing through the left end point VP1.

  Further, as shown in FIG. 9, when the head center position 51DC of the driver 51 is located in the second region (see FIG. 2), that is, an image that the driver 51 can see from the passenger seat G side of the multi-view display 4. , The angle θ1 of the optical axis AX is larger than the angle θ2 of the center line LA. In this case (θ1> θ2), the imaging plane CP must be set at a position passing through the right end point VP2. That is, when the photographing plane CP is set at a position passing through the left end point VP1, the photographing plane CP is set at a position farther from the vehicle C than the driver-side virtual plane VP. For this reason, the image processor 20 sets the imaging plane CP at a position that passes through the right end point VP2 of the driver-side virtual plane VP.

  At this time, as shown in FIG. 9, since the left end point VP1 and the imaging plane CP are separated from each other, an image cannot be displayed on the multi-view display 4 in the blind spot frame area A1 blocked by the pillar P. A non-display area B1 is generated. The non-display area B1 is actually a small area, but when the video is displayed on the multi-view display 4, the video displayed on the multi-view display 4 and the actual image that can be seen through the windows W1 and W2 are displayed. The continuity with the background is slightly impaired. On the other hand, since the right end point VP2 is in contact with the imaging plane CP, the non-display area B1 does not occur, and the continuity of the video displayed on the multi-view display 4 is higher than when the non-display area B1 exists at both ends. It becomes good.

  When the angle θ1 of the optical axis AX of the camera 6 and the angle θ2 of the center line LA are equal, that is, when the optical axis AX and the center line LA overlap, the imaging plane CP and the driver side virtual plane VP are Since they can be overlapped, the photographing plane CP is set at the position of the driver-side virtual plane VP.

  Subsequently, in S15, the control unit 10 determines each relative distance from the position detection sensors 8A to 8C arranged around the passenger 52 of the passenger seat G to the head 52D of the passenger 52 of the passenger seat G. The head center position 52DC of the passenger 52 in the passenger seat G is calculated and stored in the main memory 11.

  In S <b> 16, the image processor 20 reads the head center position 52 </ b> DC of the passenger 52 (passenger) in the passenger seat G from the main memory 11. Then, as shown in FIG. 6, the image processor 20 projects the projection position 64 on the road surface J of the head center position 52DC of the passenger 52 (passenger) of the passenger seat G and the road surface J at the lower end of the pillar P. Based on the map drawing data 17, a position coordinate of a predetermined distance (approximately 8 m in the first embodiment) from the projection position 64 in the direction toward the pillar P on the straight line connecting the center position 62 of projection to the screen. And stored in the main memory 11 as the passenger side reference position QC.

  Then, the image processor 20 determines the position of the passenger-side virtual plane WP based on the coordinates of the passenger-side reference point QC and the head center position 52DC of the passenger 52 of the passenger seat G. The passenger-side virtual plane WP is a plane on which an image taken by the camera 6 is projected. The object on the passenger-side virtual plane WP is displayed without being shifted or tilted when the image data IM photographed by the camera 6 is coordinate-converted on the passenger-side virtual plane WP. There is.

  Then, as shown in FIG. 8, the image processor 20 makes the passenger-side virtual plane WP orthogonal to the center line LB connecting the projection position 64 of the head center position 52DC and the passenger-side reference position QC, and The passenger's seat in which the photographing plane CP set in S14 is defined by the tangents L11 and L12 passing through the head center position 52DC of the passenger 52 (passenger) of the passenger seat G and the end points P1 and P2 of the pillar P, respectively. The left end point NP1 and the right end point NP2 intersecting the blind spot area A2 of the passenger 52 of G are set to a position passing through the right end point NP2 at a position far from the vehicle C. Thereby, the image processor 20 sets the passenger-side virtual plane WP at a position farther from the vehicle C than the imaging plane CP. Further, the right end point WP2 of the passenger-side virtual plane WP coincides with the right end point NP2 that intersects the tangent line L12 of the imaging plane CP. Further, the left end point WP1 of the passenger-side virtual plane WP is a point where the passenger-side virtual plane WP and the tangent line L11 intersect.

  Subsequently, in S <b> 17, the image processor 20 captures an image with the focus of the camera 6 on the imaging surface CP via the video data input unit 22 and acquires image data IM.

  In S18, as shown in FIGS. 7 and 9, the image processor 20 trims the projection portion CP1 projected on the driver-side virtual plane VP in the acquired image data IM, and the pillar blind spot data BD1. Is generated. That is, as shown in FIGS. 7 and 9, the image processor 20 trims the projection portion CP1 between the straight lines L3 and L4 connecting the camera position 6A and the end points VP1 and VP2 in the image data IM. . Data obtained by trimming the projection portion CP1 becomes pillar blind spot data BD1.

  When the pillar blind spot data BD1 is acquired, the image processor 20 projects and converts the pillar blind spot data BD1 onto the driver-side virtual plane VP. This projection conversion is a process of performing coordinate conversion of each pixel of the projection portion CP1 to each pixel on the driver-side virtual plane VP by projecting around the head center position 51DC of the driver 51. Driver-side image data PD1 that is an image viewed from the viewpoint of the driver 51 is generated by the conversion.

  Subsequently, in S19, as shown in FIGS. 8 and 10, the image processor 20 trims the projection portion CP2 to be projected onto the passenger-side virtual plane WP from the acquired image data IM, and the pillar blind spot data. BD2 is generated. That is, as shown in FIGS. 8 and 10, the image processor 20 trims the projection portion CP2 between the straight lines L13 and L14 connecting the camera position 6A and the end points WP1 and WP2 in the image data IM. . Data obtained by trimming the projection part CP2 becomes pillar blind spot data BD2.

  When the pillar blind spot data BD2 is acquired, the image processor 20 projects and converts the pillar blind spot data BD2 onto the passenger-side virtual plane WP. This projection conversion is a process of performing coordinate conversion of each pixel of the projection portion CP2 to each pixel on the passenger-side virtual plane WP by projecting around the head center position 52DC of the passenger 52 of the passenger seat G. By this coordinate conversion, passenger-side image data PD2 that is an image viewed from the viewpoint of the passenger 52 of the passenger seat G is generated.

  In S <b> 20, the image processor 20 projects and converts the driver-side image data PD <b> 1 on the screen of the multi-view display 4. In this projection conversion, each pixel of the driver-side image data PD1 is converted into a pixel of the multi-view display 4 by projecting around the head center position 51DC of the driver 51, and the multi-view display 4 is driven. Processing for generating driver-side display data OD1 that is an image for the seat F side is performed. This is because the driver side image data PD1 that is perpendicular to the center line LA, that is, the line of sight of the driver 51 is directly displayed on the multi-view display 4 that is not installed perpendicular to the line of sight of the driver 51. When displayed, the image appears distorted from the viewpoint of the driver 51, and this is a process for preventing this.

  Similarly, the image processor 20 projects and converts the passenger-side image data PD2 onto the screen of the multi-view display 4. In this projection conversion, each pixel of the passenger-side image data PD2 is converted into a pixel of the multi-view display 4 by projecting around the head center position 52DC of the passenger 52 in the passenger seat G, and the multi-view display 4 performs processing for generating passenger-side display data OD2 that is an image for the passenger seat G side.

  Subsequently, in S <b> 21, the image processor 20 reads the head center position 51 </ b> DC of the driver 51 from the main memory 11. Then, as shown in FIG. 2, the image processor 20 is configured such that the head center position 51DC of the driver 51 is positioned in the first area on the driver's seat F side in the coordinate system with the center of the multi-view display 4 as the origin. Determination processing is executed to determine whether or not the driver 51 is in the posture of viewing the video for the driver's seat F side of the multi-view display 4.

  When the head center position 51DC of the driver 51 is located in the first region on the driver's seat F side, that is, the posture of the driver 51 displays an image for the driver's seat F side of the multi-view display 4. In the case of the viewing posture (S21: YES), the image processor 20 proceeds to the process of S22. In S22, the image processor 20 outputs the driver side display data OD1 generated in S20 to the multi-view display 4 as video data for the driver's seat F side, and the passenger side display data generated in S20. OD2 is output to the multi-view display 4 as video data for the passenger seat G side, and the process ends. As a result, the driver 51 and the passenger 52 in the passenger seat G can see different images of the blind spot area blocked by the pillar P when viewed from the respective directions.

  On the other hand, when the head center position 51DC of the driver 51 is located in the second region on the passenger seat G side, that is, the posture of the driver 51 is turned forward, the passenger seat G of the multi-view display 4 is displayed. If the posture is to watch the video for the side (S21: NO), the image processor 20 proceeds to the process of S23. In S23, the image processor 20 outputs the driver-side display data OD1 generated in S20 to the multi-view display 4 as video data for the driver's seat F and video data for the passenger's seat G, and performs the processing. Exit. As a result, the driver 51 and the passenger 52 in the passenger seat G can see the same image of the blind spot area blocked by the pillar P when viewed from the direction of the driver 51.

  As explained in detail above, in the driving support system 1 according to the first embodiment, when the head position 51D of the driver 51 is in the first region shown in FIG. 2 (S21: YES), the driver 51 and An image of each blind spot area A1, A2 generated by the pillar P of the passenger 52 (passenger) in the passenger seat G can be displayed on the multi-view display 4 attached to the inside of the pillar P as an image viewed from each direction. It becomes possible. For this reason, it becomes possible for the driver 51 and the passenger 52 in the passenger seat G to display the images of the blind spot areas A1 and A2 that do not feel strange on the multi-view display 4 attached to the pillar P.

  Further, when the head position 51D of the driver 51 is at a position where an image viewed from the direction of the passenger 52 of the passenger seat G of the multi-view display 4 can be seen, that is, in the second region shown in FIG. (S21: NO), the image data of the projection portion CP1 defined by the blind spot area A1 generated by the pillar P of the driver 51 is used as the video data for the driver's seat F side and the video data for the passenger seat G side of the multi-view display 4. Display as. Therefore, the driver 51 and the passenger 52 in the passenger seat G can both display an image of the blind spot area A1 that does not feel uncomfortable on the multi-view display 4 attached to the pillar P.

  In addition, since the photographing plane CP of the camera 6 is set to a position passing through the left end point VP1 or the right end point VP2 of the driver side virtual plane VP defined by the blind spot area A1 of the driver 51, the driver of the multi-view display 4 is set. The video in the direction viewed from 51 has high continuity, and a clear and natural image can be displayed. Further, the image data of the imaging plane CP of the portion defined by the blind spot area A2 of the passenger 52 (passenger) of the passenger seat G is extracted from the image data of the imaging plane CP, and the image of the passenger seat G of the multi-view display 4 is extracted. Since it is displayed as an image in the direction viewed from the occupant 52, the image in the direction viewed from the occupant 52 of the passenger seat G of the multi-view display 4 is also highly continuous and can display a clear and natural image. Become. Therefore, the driver 51 and the passenger 52 in the passenger seat G can both display an image that does not feel strange on the multi-view display 4 attached to the pillar P.

Further, since the shooting plane CP of the camera 6 is set at a position closer to the vehicle C than the driver-side virtual plane VP, the surplus area of the image on the shooting plane CP shot by the camera 6 is not focused, and the multi-view The video in the direction viewed from the driver of the display 4 has higher continuity and can display a clear and natural image.
Further, since the passenger-side virtual plane WP is set at a position farther from the vehicle C than the shooting plane CP of the camera 6, the surplus area of the image on the shooting plane CP shot by the camera 6 is not focused, and the multi-view The image in the direction viewed from the passenger 52 of the passenger seat G of the display 4 has a higher continuity and can display a clear and natural image.

Further, according to the azimuth of the optical axis AX of the camera 6 and the azimuth of the center line LA connecting the projection position 61 of the head position 51D of the driver 51 onto the road surface J and the driver side reference position PC. The end points VP1 and VP2 through which the CP passes are changed. For this reason, it is possible to always set the photographing surface CP in front of the driver-side virtual plane VP.
Further, both ends of the passenger seat G that are perpendicular to the center line LB that connects the head position 52D of the passenger 52 and the passenger-side reference position QC, and the imaging plane CP intersects the blind spot area A2 of the passenger 52 of the passenger seat G. The passenger-side virtual plane WP is set at a position passing through the right end point WP2 far from the vehicle C among WP1 and WP2. For this reason, it is possible to always set the imaging plane CP in front of the passenger-side virtual plane WP.

Next, the driving support system 111 according to the second embodiment will be described with reference to FIGS. 11 to 13.
In the following description, the same reference numerals as those of the driving support system 1 according to the first embodiment shown in FIGS. 1 to 10 denote the same or corresponding parts as those of the driving support system 1 according to the first embodiment. It is.

The schematic configuration and control configuration of the driving support system 111 according to the second embodiment are substantially the same as those of the driving support system 1 according to the first embodiment.
However, as shown in FIG. 11, the driving support unit 2 executes the process of S111 instead of the process of S11, and the driving support unit 2 performs the processes of S120 to S122 following the process of S19. The driving support system 1 according to the first embodiment is different in that the process is executed.

  Hereinafter, display processing for displaying images of each blind spot area by the pillar P of the driver 51 and the passenger 52 of the passenger seat G on the multi-view display 4 by the driving support unit 2 of the driving support system 111 according to the second embodiment will be described with reference to FIGS. This will be described with reference to FIG.

  11 and 12 show images of the blind spot areas A1 and A2 by the driver P of the driver 51 and the passenger 52 of the passenger seat G on the multi-view display 4 by the driving support unit 2 of the driving support system 111 according to the second embodiment. It is a flowchart which shows the display process to display. FIG. 13 is an explanatory diagram of the driver-side virtual plane VP and the imaging plane CP on the driver seat F side, and the passenger-side virtual plane WP on the passenger seat G side and the imaging plane CP in front of a predetermined distance. FIG. 13 is a diagram showing a case where the angle θ1 of the optical axis AX of the camera 6 is equal to the angle θ2 of the center line LA, that is, a case where the optical axis AX and the center line LA overlap. In this case, since the photographing surface CP and the driver-side virtual plane VP can be overlapped, the photographing surface CP is set at the position of the driver-side virtual plane VP.

  As shown in FIGS. 11 and 12, first, in S <b> 111, the control unit 10 starts photographing with the camera 6 via the video data input unit 22. Further, the image data IM photographed by the camera 6 is sequentially output to the image processor 20. The image processor 20 stores the image data IM captured by the camera 6 in the main memory 11 every time it travels a predetermined distance (for example, about 1 m to 3 m).

In step S112, the control unit 10 executes the process in step S12.
Subsequently, in S113 to S119, the image processor 20 executes the processes of S13 to S19. In S <b> 114, the image processor 20 stores the imaging plane CP set for each predetermined distance in the main memory 11.

  Thereafter, in S120, the image processor 20 trims the projection portion CP2 projected on the passenger-side virtual plane WP in the image data IM acquired in S119, and determines whether or not the pillar blind spot data BD2 can be generated, that is, Determination processing for determining whether or not the photographing surface CP of the camera 6 is wide enough to intersect with the tangent L12 passing through the head center position 52DC of the passenger 52 (passenger) of the passenger seat G and the end point P2 of the pillar P. Execute.

  Then, in the image data IM acquired in S119, the projection part CP2 projected onto the passenger-side virtual plane WP can be trimmed to generate the pillar blind spot data BD2, that is, the imaging plane CP of the camera 6 is When the range is wide enough to intersect the tangent L12 passing through the head center position 52DC of the passenger 52 (passenger) of the passenger seat G and the end point P2 of the pillar P (S120: YES), the image processor 20 performs the process of S123. Transition. Then, in S123 to S126, the image processor 20 ends the process after executing the processes of S20 to S23.

  On the other hand, in the image data IM acquired in S119, the projection part CP2 projected onto the passenger-side virtual plane WP cannot be trimmed to generate the pillar blind spot data BD2, that is, the imaging plane CP of the camera 6 is In the case where the head center position 52DC of the passenger 52 (passenger) in the passenger seat G does not intersect the tangent L12 passing through the end point P2 of the pillar P (S120: NO), the image processor 20 displays the image of the camera 6 It is determined that the angle is small and the shooting range is narrow, and the process proceeds to S121. In S121, the image processor 20 was photographed by the photographing plane CP set in front of a predetermined distance (for example, about 1 m to 3 m) from the current position of the vehicle C and the camera 6 corresponding to the photographing plane CP. Image data IM is acquired from the main memory 11.

  In S122, the image processor 20 is set in front of a predetermined distance (for example, about 1 m to 3 m) from the current position of the vehicle C instead of the passenger-side virtual plane WP as shown in FIG. The passenger side virtual plane XP is set based on the photographing plane CP. The passenger-side virtual plane XP is a plane on which an image photographed by the camera 6 is projected in front of a predetermined distance (for example, about 1 m to 3 m) from the current position of the vehicle C. Further, the object on the passenger-side virtual plane XP is displayed without being shifted or tilted when the image data IM photographed by the camera 6 is coordinate-converted on the passenger-side virtual plane XP. There is.

  Specifically, as shown in FIG. 13, the image processor 20 orthogonally crosses the passenger-side virtual plane XP with a center line LB connecting the projection position 64 of the head center position 52DC and the passenger-side reference position QC. In addition, the imaging plane CP set in front of a predetermined distance (for example, about 1 m to 3 m) from the current position of the vehicle C is the head center position 52DC of the passenger 52 (passenger) in the passenger seat G. The right end point far from the vehicle C among the left end point NP1 and the right end point NP2 intersecting the blind spot area A2 of the passenger 52 of the passenger seat G defined by the tangents L11, L12 passing through the end points P1, P2 of the pillar P Set to a position that passes through NP2. Further, the right end point XP2 of the passenger-side virtual plane XP coincides with the right end point NP2 that intersects the tangent line L12 of the imaging surface CP. Further, the left end point XP1 of the passenger-side virtual plane XP is a point where the passenger-side virtual plane XP intersects the tangent L11.

  Subsequently, as shown in FIG. 13, the image processor 20 is a passenger in the image data IM photographed by the camera 6 before a predetermined distance (for example, about 1 m to 3 m) from the current position of the vehicle C. The projection part CP2 projected onto the side virtual plane XP is trimmed to generate pillar blind spot data BD2. That is, as shown in FIG. 13, the image processor 20 trims the projection portion CP2 between the tangents L11 and L12 in the image data IM before a predetermined distance. Data obtained by trimming the projection part CP2 becomes pillar blind spot data BD2.

  When the pillar blind spot data BD2 is acquired, the image processor 20 projects and converts the pillar blind spot data BD2 onto the passenger-side virtual plane XP. This projection conversion is a process of performing coordinate conversion of each pixel of the projection portion CP2 to each pixel on the passenger-side virtual plane XP by projecting around the head center position 52DC of the passenger 52 of the passenger seat G. After generating the passenger side image data PD2 which is an image viewed from the viewpoint of the passenger 52 of the passenger seat G by this coordinate conversion, the process proceeds to S123. Then, in S123 to S126, the image processor 20 ends the process after executing the processes of S20 to S23.

As described above in detail, the driving support system 111 according to the second embodiment has the following effects in addition to the effects of the driving support system 1 according to the first embodiment.
The image processor 20 trims the projection portion CP2 projected on the passenger-side virtual plane WP in the acquired image data IM and cannot generate the pillar blind spot data BD2, that is, the imaging plane CP of the camera 6 is When the head center position 52DC of the passenger 52 (passenger) in the passenger seat G does not intersect the tangent L12 passing through the end point P2 of the pillar P (S120: NO), a predetermined distance from the current position of the vehicle C ( For example, it is about 1 m to 3 m.) From the imaging plane CP set in front and the image data IM captured by the camera 6 corresponding to the imaging plane CP, the passenger-side virtual plane XP is set, The passenger side image data PD2 can be created (S121 to S122).

  For this reason, even when the angle of view of the camera 6 is small and the imaging range is narrow, the image data IM of the imaging surface CP of the portion partitioned by the blind spot area A2 due to the pillar P of the passenger 52 of the passenger seat G is extracted. The multi-view display 4 can be displayed as an image in the direction viewed from the passenger 52 side of the passenger seat G, and the image in the direction viewed from the passenger 52 side of the passenger seat G of the multi-view display 4 is also continuous. A high, clear and natural image can be displayed. Therefore, even when the angle of view of the camera 6 is small, the driver 51 and the passenger 52 (passenger) in the passenger seat G display an image that does not feel uncomfortable on the multi-view display 4 provided in the pillar P. Is possible.

  In addition, this invention is not limited to the said Example 1 and Example 2, Of course, various improvement and deformation | transformation are possible within the range which does not deviate from the summary of this invention. For example, the following may be used.

  (A) If the multi-view display 4 is not limited to an LCD (Liquid Crystal Display), that is, a liquid crystal display, but a highly flexible, for example, low-temperature polysilicon TFT (ThinFilm Transistor) liquid crystal display, the flat display shown in FIG. It is not limited to a simple shape, and can be used by being attached along the curved surface of the inner side surface PA of the pillar P. There is an advantage that the vehicle interior space can be expanded by suppressing the amount of protrusion to the vehicle interior side accordingly.

  (B) The camera 6 is attached to the upper right corner as viewed from the passenger compartment of the front window W1 of the vehicle C. The blind spot areas A1 and A2 generated by the pillars P of the driver 51 and the passenger 52 (passenger) of the passenger seat G may be provided anywhere.

  (C) In the second embodiment, after executing the process of step 119, the process of step 121 may be performed without executing the process of step 120. Thereby, in addition to the effect of the driving support system 111 according to the second embodiment, the processing speed of the image processor 20 can be increased.

1 is a block diagram illustrating a configuration of a driving support system according to a first embodiment. It is explanatory drawing of the position detection sensor which detects a driver | operator's head and the passenger | crew's head of a passenger seat. It is explanatory drawing of the attachment position of a camera. It is explanatory drawing of the attachment position of a camera and a multi view display. It is a flowchart which shows the display process which displays the image | video of each blind spot area | region by the driver | operator and passenger's passenger's pillar on a multi view display by a driving assistance unit. It is explanatory drawing of a driver | operator side reference point and a passenger side reference point. It is explanatory drawing of the driver | operator side virtual plane of a driver's seat side, and an imaging surface. It is explanatory drawing of the passenger side virtual plane and imaging surface of the passenger seat side. It is explanatory drawing of the driver | operator side virtual plane of a driver's seat side, and an imaging surface. It is explanatory drawing of the passenger side virtual plane and imaging surface of the passenger seat side. 12 is a flowchart illustrating display processing for displaying images of each blind spot area by pillars of drivers and passengers on a multi-view display by the driving support unit of the driving support system according to the second embodiment. It is a flowchart which shows the display process which displays the image | video of the blind spot area | region by the driver | operator and passenger's passenger's pillar on a multi view display by a driving assistance unit. It is explanatory drawing of the driver | operator side virtual plane and imaging | photography surface of a driver's seat side, the passenger side virtual plane of a passenger seat side, and the imaging | photography surface before a predetermined distance.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Driving assistance system 2 Driving assistance unit 4 Multi view display 6 Camera 10 Control part 11 Main memory 12 ROM
20 Image processor 22 Video data input part 51 Driver 52 Crew (passenger)
61, 64 Projection position A1, A2 Blind spot area AX Optical axis CP Imaging surface F Driver's seat G Passenger's seat L1, L2, L11, L12 Tangent LA, LB Center line NP1 Left end point NP2 Right end point P Front pillar (pillar)
PC Driver side reference position QC Passenger side reference position VP Driver side virtual plane VP1 Left end point VP2 Right end point WP, XP Passenger side virtual plane WP1, XP1 Left end point WP2, XP2 Right end point

Claims (11)

Image data acquisition means for acquiring image data including a driver's blind spot area generated by a front pillar of the vehicle from imaging means for imaging the outside of the vehicle;
Driver-side extraction means for extracting image data of a portion partitioned by the driver's blind spot area from the image data;
Passenger side extraction means for extracting image data of a portion partitioned by the blind spot area of the passenger from the image data;
Image data extracted by the driver-side extraction means on the multi-view display means provided inside the front pillar and capable of displaying individual images at least when viewed from each direction of the driver and passengers Multi-view display control means for controlling the image data extracted by the passenger-side extraction means to be displayed as an image of the direction seen from the passenger ,
A driving support apparatus comprising: The multi-view display control means includes head position determination means for determining whether or not the driver's head position is at a position where an image viewed from the passenger's direction of the multi-view display means can be seen. ,
When the multi-view display control means determines that the driver's head position is at a position where the image viewed from the passenger's direction of the multi-view display means can be seen, the driver-side extraction means The driving support device according to claim 1, wherein the extracted image data is controlled to be displayed as an image in each direction viewed from the driver and a passenger. Image data storage means for sequentially storing the image data acquired by the image data acquisition means;
Extraction determination means for determining whether or not image data of a portion partitioned by the blind spot area of the fellow passenger can be extracted from the image data acquired by the image data acquisition means;
With
When it is determined that the image data of a portion defined by the passenger's blind spot area cannot be extracted from the image data acquired by the image data acquisition unit, the passenger-side extraction unit determines a predetermined value from the own vehicle position. The driving support device according to claim 1 or 2, wherein image data of a portion partitioned by the blind spot area of the fellow passenger is extracted from image data stored at a position before the distance. Driver-side virtual plane setting means for setting a driver-side virtual plane for projecting image data of the driver's blind spot area of the image data acquired by the image data acquisition means;
A photographing surface setting means for setting a photographing surface of the photographing means to a position passing through an end point of the driver-side virtual plane defined by the driver's blind spot area;
Passenger-side virtual plane setting means for setting a passenger-side virtual plane for projecting image data of the blind spot area of the passenger among the image data acquired by the image data acquisition means;
A focus control means for controlling the photographing surface set by the photographing surface setting means to obtain the image data by focusing the photographing means;
Driver-side image processing means for coordinate-converting the image data extracted by the driver-side extraction means into the driver-side virtual plane;
Passenger-side image processing means for coordinate-converting the image data extracted by the passenger-side extraction means into the passenger-side virtual plane;
With
The multi-view display control means displays the image data coordinate-transformed on the driver-side virtual plane by the driver-side image processing means as an image viewed from the driver's seat, and the passenger-side image processing The driving support device according to any one of claims 1 to 3, wherein the image data coordinate-converted to the passenger-side virtual plane by means is displayed as an image in a direction viewed from the passenger.   The driving support apparatus according to claim 4, wherein the imaging plane setting unit sets the imaging plane of the imaging unit closer to the vehicle than the driver-side virtual plane.   6. The driving support apparatus according to claim 5, wherein the passenger-side virtual plane setting unit sets the passenger-side virtual plane at a position farther from the vehicle than the imaging plane of the imaging unit. The driver-side virtual plane setting means is configured so that the driver-side reference on the road surface in the blind spot area of the driver based on the head position of the driver and the projection position on the road surface of the front pillar. After setting the position, set the driver-side virtual plane at a position that passes the driver-side reference position,
The imaging plane setting means is based on the azimuth of the optical axis of the imaging means and the azimuth of a straight line connecting the projected position of the driver's head position on the road surface and the driver-side reference position. The driving support device according to any one of claims 4 to 6, wherein an end point of the driver-side virtual plane where planes intersect is selected.   The passenger-side virtual plane setting means is configured to provide a passenger-side reference on the road surface in the blind spot area of the passenger based on the head position of the passenger and the projected position of the front pillar on each road surface. After setting the position, it is orthogonal to a straight line connecting the passenger's head position and the passenger-side reference position, and the imaging plane is far from the vehicle at both end points intersecting the blind spot area of the passenger. The driving support device according to claim 7, wherein the passenger-side virtual plane is set at a position passing through an end point of the position. The image data storage means sequentially stores image data of the photographing surface,
The extraction determining means determines whether or not image data of a photographing surface of a portion partitioned by the blind spot area of the fellow passenger can be extracted from the image data of the photographing surface set by the photographing surface setting means,
When it is determined that the passenger side extraction means cannot extract the image data of the photographing surface of the portion defined by the blind spot area of the passenger from the image data of the photographing surface set by the photographing surface setting means. Is to extract the image data of the imaging surface of the portion partitioned by the blind spot area of the passenger from the image data of the imaging surface stored at a position a predetermined distance before the own vehicle position,
9. The passenger-side image processing unit performs coordinate conversion of the image data extracted by the passenger-side extraction unit into the passenger-side virtual plane, according to any one of claims 4 to 8. Driving assistance device. An image data acquisition step of acquiring image data including a blind spot area of a driver generated by a front pillar of the vehicle from an imaging unit that images the outside of the vehicle;
A driver-side extraction step of extracting image data of a portion partitioned by the driver's blind spot area from the image data;
Passenger side extraction step of extracting the image data of the portion partitioned by the blind spot area of the passenger from the image data,
Image data extracted in the driver-side extraction step in multi-view display means provided inside the front pillar and capable of displaying individual images at least when viewed from each direction of the driver and the passenger. A multi-view display control process for controlling the image data extracted in the passenger-side extraction process to be displayed as an image of the direction viewed from the passenger, ,
A driving support method comprising: On the computer,
An image data acquisition step of acquiring image data including a blind spot area of a driver generated by a front pillar of the vehicle from an imaging unit that images the outside of the vehicle;
A driver-side extraction step of extracting image data of a portion partitioned by the driver's blind spot area from the image data;
A passenger-side extraction step of extracting image data of a portion partitioned by the passenger's blind spot area from the image data;
The image data extracted in the driver-side extraction step is provided in a multi-view display means provided inside the front pillar and capable of displaying individual images at least when viewed from each direction of the driver and the passenger. A multi-view display control process for controlling the image data extracted in the passenger-side extraction process to be displayed as an image of the direction viewed from the passenger, ,
A program for running

Images