JP2008236711A - Driving support method and driving support device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a driving support method and a driving support device capable of projecting onto a pillar an image for which a driver feels quite normal. <P>SOLUTION: In a driving support unit 2 that uses a camera 6 attached to a vehicle for imaging a blind spot area created by a pillar of the vehicle and projects an image taken by the camera 6 on an inner side of the pillar, a virtual plane on which image data is projected is set, and an imaged plane of the camera 6 is set at a predetermined position for reducing a non-displayed area or an excess area that is generated by a deviation between the virtual plane and the imaged plane of the camera 6. Image data input from the camera 6 is coordinate-transformed onto the virtual plane, and an image corresponding to the blind spot area caused by the pillar is displayed on the inner side of the pillar, based on the coordinate-transformed image data on the virtual plane. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a driving support method and a driving support device.

  2. Description of the Related Art Conventionally, as an apparatus that supports safe driving, an in-vehicle system that captures an area that is a blind spot of a driver with an in-vehicle camera and displays it on a display or the like has been developed. As one of such systems, a system has been proposed in which a blind spot area caused by a front pillar of a vehicle is photographed with an in-vehicle camera and a photographed image is displayed inside the front pillar. The front pillar is a left and right column that supports the front window and roof.It is located diagonally forward when viewed from the driver seated in the driver's seat and partially blocks the driver's field of view. This is a member that must be secured.

  As shown in FIG. 12, the system described above is attached to the vehicle body and can capture the image capturing area 106, an image processor that performs image processing on a video signal output from the camera 100, and an image inside the front pillar 101. Projectors (both not shown) and the like are provided. According to this, as seen from the driver position 102, the external background can be seen through the front pillar 101. Therefore, at the intersection or the like, the road shape on the front side of the vehicle or the front side is present. Obstacles can be confirmed.

When projecting a video signal photographed by the camera 100 onto the pillar 101, since the viewing angle of the camera 100 does not match the viewing angle of the driver, the image projected on the pillar 101 is being viewed by the driver through the window. There is a problem of tilting or shifting. On the other hand, in Patent Document 1, an image captured by a camera is projected and converted to a virtual plane (virtual screen surface) set to match the viewing angle of the driver.
JP 2005-184225 A

  However, when determining the focal position of the camera 100, for example, as shown in FIG. 12, even if the imaging surface 104 indicating the focal position is set in front of the virtual plane 103 and close to the virtual plane 103, As long as the imaging surface 104 and the virtual plane 103 are separated from each other, areas 107 and 108 that are not projected onto the virtual plane 103 are generated. Further, as shown in FIG. 13, when the imaging plane 104 is set at a position farther from the driver position 102 than the virtual plane 103, areas 109 and 110 between the imaging plane 104 and the virtual plane 103 are projected onto the pillars. Waste.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a driving support method and a driving support device that can project an image on which a driver does not feel uncomfortable on a pillar.

In order to solve the above-mentioned problem, the invention according to claim 1 shoots a blind spot area caused by a pillar of the vehicle using an imaging device attached to the vehicle, and displays an image corresponding to the blind spot area. In the driving support method for displaying inside the pillar, the virtual plane for projecting the image data input from the imaging apparatus is set, and the imaging plane of the imaging apparatus is divided by a blind spot area generated by the pillar Is set to a position that passes through the end point of the image, the image data focused on the imaging surface is acquired, the image data is coordinate-converted to the virtual plane, and based on the image data coordinate-converted to the virtual plane, The gist is to display an image corresponding to a blind spot region caused by the pillar inside the pillar.

  According to a second aspect of the present invention, driving assistance for photographing a blind spot area caused by a pillar of the vehicle by using a photographing device attached to the vehicle and displaying an image photographed by the photographing device inside the pillar. In the apparatus, a virtual plane setting unit that sets a virtual plane on which image data input from the imaging apparatus is projected, and an imaging plane of the imaging apparatus passes through an end point of the virtual plane defined by a blind spot area generated by the pillar. An imaging plane setting means for setting the position, an image processing means for acquiring the image data focused on the imaging plane, and coordinate-converting the image data to the virtual plane, and coordinate-converted to the virtual plane An output control means for displaying an image of a blind spot area generated by the pillar on the inner side of the pillar based on the image data. That.

  According to a third aspect of the present invention, in the driving support device according to the second aspect, the photographing surface setting means sets the photographing surface of the photographing device closer to the vehicle than the virtual plane. The gist.

  According to a fourth aspect of the present invention, in the driving support apparatus according to the second or third aspect, the virtual plane setting means acquires a position of a drawing reference object on a road surface around the vehicle, and the drawing reference object The virtual plane is set at a position passing through a position, and the imaging plane setting means includes an orientation of an optical axis of the imaging apparatus, and an orientation of a straight line connecting the head position of the driver of the vehicle and the position of the drawing reference object. Based on the above, the gist is to select the end point of the virtual plane where the imaging planes intersect.

  According to the first aspect of the present invention, the photographing surface of the photographing device is set to a position passing through the end point of the virtual plane. For this reason, a non-display area that is not projected onto the pillar and a surplus area of the captured image can be reduced, and a clear, natural image with high continuity can be displayed on the pillar.

  According to the invention described in claim 2, the driving support device sets the photographing surface of the photographing device to a position passing through the end point of the virtual plane. For this reason, a non-display area that is not projected onto the pillar and a surplus area of the captured image can be reduced, and a clear, natural image with high continuity can be displayed on the pillar.

  According to the third aspect of the present invention, since the photographing surface of the photographing device is set at a position closer to the vehicle than the virtual plane, the surplus area is not in focus, has high continuity, and is a clear and natural image. Can be displayed.

  According to the fourth aspect of the present invention, the end point through which the imaging surface passes is changed according to the orientation of the optical axis of the imaging device and the orientation of the straight line connecting the head position of the driver and the drawing reference object. For this reason, the imaging plane can always be set in front of the virtual plane.

Hereinafter, an embodiment embodying the present invention will be described with reference to FIGS. FIG. 1 is a block diagram illustrating a configuration of a driving support system 1 mounted on an automobile.
The driving support system 1 is mounted on a vehicle C (see FIG. 3), and as shown in FIG. 1, a driving support unit 2 as a driving support device, a display 3, a projector 4 as a projection device, a speaker 5, and an imaging device. The camera 6 includes first to third position detection sensors 8a to 8c.

The driving support unit 2 includes a control unit 10, a nonvolatile main memory 11, a ROM 12, and a GPS receiving unit 13. The control unit 10 is a CPU, MPU, ASIC, or the like, and R
The main control of each process is performed according to the driving support program stored in the OM 12. The main memory 11 temporarily stores the calculation result of the control unit 10.

  The control unit 10 acquires satellite orbit information and time information received by the GPS receiving unit 13 from GPS satellites, and calculates the absolute position of the host vehicle by radio wave navigation. Further, the control unit 10 inputs vehicle speed pulses and angular velocities from the vehicle speed sensor 30 and the gyro 31 provided in the vehicle C via the vehicle side I / F unit 14 of the driving support unit 2. And the control part 10 calculates the relative position from a reference position by the autonomous navigation using a vehicle speed pulse and an angular velocity, and pinpoints the own vehicle position combining with the absolute position calculated by the radio wave navigation.

  The driving support unit 2 includes a geographic data storage unit 15. The geographic data storage unit 15 is an external storage medium such as an internal hard disk or an optical disk. In the geographic data storage unit 15, each route network data (hereinafter referred to as route data 16) as map data for searching for a route to the destination, and map data for outputting the map screen 3 a to the display 3. Each map drawing data 17 is stored.

  The route data 16 is data relating to roads in the mesh dividing the whole country. The route data 16 includes data such as identifiers of each mesh, node data related to nodes indicating intersections and road endpoints, identifiers of links connecting the nodes, and link costs. The control unit 10 uses the route data 16 to search for a recommended route to the destination, and determines whether the vehicle C is approaching a guide point such as an intersection.

  The map drawing data 17 is data for drawing a road shape, a background, and the like, and is stored for each mesh obtained by dividing a map of the whole country. The map drawing data 17 stores data on road markings such as a center line, a white line that divides a roadside zone, a zebra zone, and a pedestrian crossing, and road surface installations such as traffic lights. Specifically, the types of road markings, the position coordinates of road markings, the types of road surface installation objects, the position coordinates of road surface installation objects, and the like are stored in association with the intersections and curves of each point.

  As shown in FIG. 1, the driving support unit 2 includes a map drawing processor 18. The map drawing processor 18 reads out the map drawing data 17 for drawing a map around the vehicle position from the geographic data storage unit 15, generates map output data, and displays the map screen 3a based on the map output data. Displayed on the display 3. Further, the map drawing processor 18 superimposes a vehicle position index 3b indicating the vehicle position on the map screen 3a.

  In addition, the driving support unit 2 includes an audio processor 24. The audio processor 24 has an audio file (not shown) and outputs, for example, audio from the speaker 5 that guides the route to the destination. Further, the driving support unit 2 includes an external input I / F unit 25. The external input I / F unit 25 inputs an input signal based on a user input operation from the operation switch 26 adjacent to the display 3 and the touch panel display 3 and outputs the input signal to the control unit 10.

  The driving support unit 2 includes a sensor I / F unit 23 that constitutes a detection unit. The sensor I / F unit 23 receives detection signals from the first to third position detection sensors 8a to 8c. The first to third position detection sensors 8a to 8c are constituted by ultrasonic sensors, and are attached around the driver D seated on the front seat F in the vehicle interior as shown in FIG. The first position detection sensor 8a is attached in the vicinity of a rearview mirror (not shown), and is located at a position that is substantially the same as or slightly higher than the head D1 of the driver D.

The second position detection sensor 8b is attached in the vicinity of the upper end of the door window W2 (see FIG. 3) so as to be positioned diagonally forward right of the driver D. The third position detection sensor 8 c is attached to the left side of the front seat F and inside the roof R. Ultrasonic waves transmitted from sensor heads (not shown) of the respective position detection sensors 8a to 8c are reflected on the head D1 of the driver D. Each position detection sensor 8a-8c measures the time from transmitting an ultrasonic wave until receiving a reflected wave, and calculates each relative distance to the head D1 based on the measurement time. Each calculated relative distance is output to the control unit 10 via the sensor I / F unit 23. The sensor I / F unit 23 may calculate the relative distance to the head D1 based on signals from the position detection sensors 8a to 8c.

  The control unit 10 has a head movement range in which the head D1 of a standard body type driver D can move in the state of being seated in the driver's seat, and the relative positions detected by the first to third position detection sensors 8a to 8c. Based on the distance, the head center position Dc as the head position is obtained by a known method using triangulation or the like.

  As shown in FIG. 1, the driving support unit 2 includes a video data input unit 22 and an image processor 20 as a virtual plane setting unit, a photographing plane setting unit, an image processing unit, and an output control unit. The video data input unit 22 drives the camera 6 provided in the vehicle C based on the control of the control unit 10 and inputs the image data IM taken by the camera 6.

  The camera 6 is a camera that captures a color image, and includes an optical mechanism including a lens, a mirror, and the like, a CCD image sensor (none of which is shown), an autofocus mechanism, and the like. As shown in FIG. 3, the camera 6 is attached to the outside of the front pillar P of the vehicle C (hereinafter simply referred to as the pillar P) with the optical axis facing the front of the vehicle C. In the present embodiment, the camera 6 is attached to the right pillar P on the driver's seat side in accordance with the driver's seat arranged on the right side of the vehicle. This camera 6 images the background of the imaging region Z1 including the right front side of the vehicle C and a part of the right side of the vehicle C.

  The image processor 20 of the driving support unit 2 acquires the image data IM from the camera 6 via the video data input unit 22. In addition, the image processor 20 trims a region blocked by the pillar P in the acquired image data IM and performs image processing for eliminating image distortion.

  More specifically, when the control unit 10 determines that the vehicle C has approached an intersection or curve based on the route data 16, the image processor 20 acquires the coordinates of the visual target that is easy for the driver D to gaze at the intersection or curve. To do. In the present embodiment, the image processor 20 acquires the coordinates of the pedestrian crossing Z as a drawing reference object based on the map drawing data 17. For example, as shown in FIG. 4, the coordinates of the reference point Pc of the pedestrian crossing Z marked on the front side of the vehicle are acquired based on the map drawing data 17. The reference point Pc is stored in advance in the map drawing data 17 or is set by the image processor 20 from the coordinates of the entire pedestrian crossing Z stored in the map drawing data 17.

When the coordinates of the reference point Pc are acquired, the position of the virtual plane VP is determined based on the coordinates of the reference point Pc and the head center position Dc of the driver D detected by the first to third position detection sensors 8a to 8c. To do. The virtual plane VP is a plane on which an image taken by the camera 6 is projected. In addition, the object on the virtual plane VP has a property that it is displayed without being shifted or tilted when the image data IM photographed by the camera 6 is coordinate-converted on the virtual plane VP. As shown in FIG. 5, the image processor 20 sets the virtual plane VP to a position that includes the reference point Pc and is perpendicular to the center line La that connects the head center position Dc and the reference point Pc. . Further, the image processor 20 provides the virtual plane VP inside the blind spot area A defined by tangent lines L1 and L2 passing through the head center position Dc of the driver D and the end points P1 and P2 of the pillar P.

  As shown in FIG. 4, the vehicle C moves from the initial position FP where the virtual plane VP is set to a position indicated by a two-dot chain line in the figure, and the relative position between the head center position Dc and the reference point Pc of the pedestrian crossing Z. Even if the distance ΔL changes, the image processor 20 sets the virtual plane VP at a position that passes through the reference point Pc and is perpendicular to the center line La. That is, the virtual plane VP is always set to a position that passes through the reference point Pc without being moved as the vehicle C moves forward. As shown in FIG. 4, when the vehicle is turning, the inclination of the center line La connecting the head center position Dc and the reference point Pc changes. Accordingly, the virtual plane VP also changes only the angle with the change. To do.

  Further, the image processor 20 determines the imaging plane CP according to the position of the virtual plane VP. The imaging plane CP indicates the focal position of the camera 6 and is represented by a plane perpendicular to the optical axis AX of the camera 6. As shown in FIG. 5, the image processor 20 sets the photographing plane CP at a position closer to the vehicle C than the virtual plane VP and the left end point VP1 or right end of the virtual plane VP defined by the blind spot area A. It is set to a position (predetermined position) that intersects at the point VP2. Here, the imaging plane CP is set to a position close to the head center position Dc with the head center position Dc of the driver D as a reference. Note that the end points VP1 and VP2 are intersections of the tangent lines L1 and L2 connecting the head center position Dc and the end points P1 and P2 of the pillar P, respectively, and the virtual plane VP.

  To describe the setting method of the imaging plane CP in detail, first, the image processor 20 compares the orientation of the optical axis AX of the camera 6 with the orientation of the center line La connecting the head center position Dc and the reference point Pc. . That is, an angle θc formed by the optical axis AX and the horizontal direction (Y arrow direction) and an angle θd formed by the center line La and the horizontal direction (Y arrow direction) are calculated. Then, the magnitude of the angle θc of the optical axis AX is compared with the magnitude of the angle θd of the center line La. The X arrow direction is a direction parallel to the longitudinal direction of the vehicle C, and the Y arrow direction is a direction orthogonal to the X arrow direction and is a vehicle width direction.

  As shown in FIG. 5, when the angle θc of the optical axis AX is larger than the angle θd of the center line La (θc> θd), in order to set the photographing surface CP at a position satisfying the above condition, the photographing surface CP is The position must pass through the right end point VP2. That is, when the imaging plane CP is set at a position passing through the left end point VP1, the position is set farther from the head center position Dc than the virtual plane VP. Therefore, the image processor 20 sets the photographing plane CP at a position that passes through the right end point VP2 of the virtual plane VP.

  Also, as shown in FIG. 6, when the angle θc of the optical axis AX is larger than the angle θd of the center line La (θc <θd), the image processor 20 sets the photographing plane CP to the left end point VP1 of the virtual plane VP. Set to a position that passes through.

  When the angle θc of the optical axis AX and the angle θd of the center line La are equal, that is, when the optical axis AX and the center line La overlap, the imaging plane CP and the virtual plane VP can be overlapped, so that the imaging plane CP is set at the position of the virtual plane VP.

When the imaging plane CP is set, the image processor 20 captures an image with the focus of the camera 6 on the imaging plane CP via the video data input unit 22 and acquires image data IM.
When the image data IM is acquired, the projection part CP1 projected onto the virtual plane VP in the image data IM is trimmed to generate pillar blind spot data BD. That is, as shown in FIG. 5, the image processor 20 trims the projection portion CP1 between the straight lines L3 and L4 connecting the camera position 6a and the end points VP1 and VP2 in the image data IM. Data obtained by trimming the projection portion CP1 becomes pillar blind spot data BD.

  When the pillar blind spot data BD is acquired, the image processor 20 projects and converts the pillar blind spot data BD onto the virtual plane VP. This projection conversion is a process of coordinate-converting each pixel of the projection portion CP1 to each pixel on the virtual plane VP.

  Further, the image processor 20 projects and converts the image projected and converted onto the virtual plane VP onto a pillar surface including the inner surface Pa of the pillar P. The pillar surface can be set based on the three-dimensional coordinates of the pillar shape 41 (see FIG. 1) stored in the ROM 12. The pillar shape 41 is data indicating the outer shape of the pillar P by a pattern or coordinates, and is different data depending on the vehicle type. Based on this pillar shape 41, the image processor 20 can acquire three-dimensional coordinates in the length direction, the vehicle width direction, and the vertical direction that indicate the outer shape of the pillar P. The width and length of the pillar P can also be acquired as data using three-dimensional coordinates.

  Subsequently, in order to project the image projected onto the pillar surface and conforming to the position and shape of the pillar P by the projector 4, the image is coordinate-transformed according to the position of the projector 4. That is, the image displayed on the inner surface Pa is distorted, enlarged, or reduced depending on the angle at which the light output from the projector 4 is incident on the inner surface Pa of the pillar P. Therefore, for example, a map or the like in which the coordinates of each pixel of the image subjected to the projection conversion and the coordinates of each pixel of the image output to the projector 4 are associated in advance is stored in the main memory 11 or the like in advance. Then, the projected image is further coordinate-converted as an output image to the projector 4. Then, projection data PD for outputting the coordinate-converted image to the projector 4 is generated.

  Further, the image processor 20 generates a video signal to be output to the projector 4 based on the mask pattern 40 (see FIGS. 1 and 7) stored in the ROM 12 for the projection data PD. As shown in FIG. 7, the mask pattern 40 is data for applying a mask to the projection data PD, and includes an image display region 40a along the inner surface shape of the pillar P and a mask 40b. The image processor 20 reads the projection data PD in the area of the image display area 40 a, and generates the output data OD for hiding the projector 4 in the area of the mask 40 b. When the output data OD is generated, the image processor 20 outputs the output data OD to the projector 4.

  As shown in FIG. 8, the projector 4 is mounted on the inner side of the pillar P on the right side of the vehicle C so that an image can be projected on the inner side of the front seat F on which the driver D sits inside the roof R. It has been. As shown in FIG. 9, a screen SC cut according to the shape of the pillar P is attached to the inner side surface Pa of the pillar P. The focus of the projector 4 is adjusted according to this screen SC. Note that the screen SC may be omitted when the inner surface Pa of the pillar P is made of a material and shape capable of receiving the projection light output from the projector 4 and displaying a clear image.

  As shown in FIG. 10, the projector 4 emits projection light L to the screen SC of the pillar P, and projects an image on the screen SC. Further, an image is not projected on the front window W1 or the door window W2 around the screen SC by the mask 40b.

  Next, the processing procedure of this embodiment will be described with reference to FIG. First, the control unit 10 of the driving support unit 2 waits for the start of a projection mode for projecting a background image inside the pillar P (step S1). For example, when the touch panel and the operation switch 26 are operated and the control unit 10 receives a mode start request via the external input I / F unit 25, it is determined to start the projection mode. Alternatively, it may be determined that the projection mode is started based on an ON signal from an ignition module (not shown).

  If it is determined that the projection mode is to be started (YES in step S1), the control unit 10 waits for the vehicle C to approach the intersection or curve based on the route data 16 (step S2). Specifically, when the control unit 10 determines that the current position of the vehicle C has entered a predetermined distance range (for example, 200 m) from an intersection including a T-junction or a curve having a predetermined curvature or more, the control unit 10 enters the intersection or the curve. Judge that approached.

  When it is determined that the vehicle has approached the intersection or curve (YES in step S2), the control unit 10 detects the head position of the driver D using the position detection sensors 8a to 8c (step S3). At this time, the control unit 10 acquires the relative distances from the position detection sensors 8a to 8c to the head D1 via the sensor I / F unit 23. Then, using each relative distance, the head center position Dc is specified based on the principle of triangulation.

  When the head center position Dc is calculated, the image processor 20 sets the virtual plane VP as described above (step S4). When the vehicle C is approaching the intersection J (see FIG. 4), the reference point Pc of the pedestrian crossing Z is acquired based on the map drawing data 17, and the virtual plane VP is set at a position passing through the reference point Pc. . When there is no pedestrian crossing Z, a road marking such as a stop line in front of the vehicle may be used. When the vehicle C is approaching a curve, the reference point Pc may be taken at a position where the curvature is large in the center line marked on the road.

  Further, the image processor 20 sets the photographing plane CP (step S5). At this time, as described above, the image processor 20 compares the angle θc of the optical axis AX of the camera 6 with the angle θd of the center line La connecting the head center position Dc and the reference point Pc. Then, as shown in FIG. 5, when the angle θc of the optical axis AX is larger than the angle θd of the center line La, as described above, the imaging plane CP is in front of the virtual plane VP, and the virtual plane VP It is set to a position passing through the right end point VP2 of the plane VP. At this time, as shown in FIG. 5, the non-display area B1 that cannot be projected onto the pillar P in the blind spot area A blocked by the pillar P due to the separation of the left end point VP1 and the imaging surface CP. Arise. The non-display area B1 is actually a small area, but when an image is displayed on the pillar P, the image displayed on the pillar P and the actual background visible through the windows W1 and W2 are continuous. The characteristics are slightly impaired. On the other hand, since the right end point VP2 is in contact with the imaging plane CP, the non-display area B1 does not occur, and the image continuity is better than when the non-display area B1 exists at both ends.

  Furthermore, since the photographing plane CP is set closer to the head center position Dc of the driver D than the virtual plane VP, there is no surplus area that is not projected on the pillar P in the image photographed in focus. .

  As shown in FIG. 6, when the angle θc of the optical axis AX is smaller than the angle θd of the center line La, the imaging plane CP is set to a position passing through the left end point VP1 of the virtual plane VP. At this time, since the left end point VP1 and the imaging plane CP are in contact, the non-display area B1 does not occur at the left end point VP1.

  When the imaging plane CP is set, the camera 6 performs imaging with the focus on the imaging plane CP based on the imaging plane CP set by the image processor 20, and acquires image data IM (step S6).

  When acquiring the image data IM, the image processor 20 extracts the projection portion CP1 projected onto the virtual plane VP from the image data IM as described above (step S7), and generates pillar blind spot data BD.

  When the pillar blind spot data BD is generated, the image processor 20 performs projection image processing on the pillar blind spot data BD (step S8). Specifically, as described above, the pillar blind spot data BD is projected and converted to the virtual plane VP set in step S4. Further, the image projected and converted to the virtual plane VP is converted according to the three-dimensional shape of the pillar P based on the pillar shape 41. In addition, the image converted according to the three-dimensional shape of the pillar P is subjected to coordinate conversion based on the position of the projector 4 to generate projection data PD. Further, based on the projection data PD and the mask pattern 40, the output data OD masking the area other than the pillar P is generated.

  When the output data OD is generated, the image processor 20 outputs the output data OD to the projector 4. As shown in FIG. 10, the projector 4 projects an image of an area blocked by the pillar P onto the screen SC provided on the inner side surface Pa of the pillar P (step S9). Since the projected image displayed on the screen SC has been projected and converted in advance to a virtual plane VP that matches the road marking such as the pedestrian crossing Z, the road marking is visually recognized through the front window W1 and the door window W2. The background is displayed without shifting or tilting. In addition, since the imaging plane CP is in contact with one of the end points VP1 and VP2 of the virtual plane VP, the non-display area B1 and the surplus area caused by the deviation between the virtual plane VP and the imaging plane CP are reduced, and the window W1, Continuity with the background seen through W2 is improved.

  When the image processor 20 projects an image on the pillar P, the control unit 10 determines whether or not the vehicle C has left the intersection J or the curve (step S10). If it is determined that the vehicle C is at or near the intersection J or a curve (NO in step S10), the process returns to step S3, and the head position detection (step S3) to image projection (step S9) are repeated. That is, an image blocked by the pillar P is projected onto the pillar P until the vehicle C leaves the intersection J or the curve.

  If it is determined that vehicle C has left the intersection J or the curve (YES in step S10), control unit 10 determines whether or not to end the projection mode (step S11). The end trigger is a mode end request by an operation of the touch panel or the operation switch 26, or an ignition module OFF signal. When these signals are input, it is determined that the projection mode has ended (YES in step S11), and the process ends.

According to the above embodiment, the following effects can be obtained.
(1) In the above embodiment, when the vehicle C approaches an intersection or curve, the image processor 20 sets the virtual plane VP at a position that passes through the reference point Pc of an object such as a road marking. In addition, the imaging plane CP of the camera 6 is set in front of the virtual plane VP and at a position passing through the end points VP1 and VP2 of the virtual plane VP. For this reason, although it is the blind spot area A obstructed by the pillar P, the non-display area B1 that cannot be projected onto the pillar P, or the surplus area that cannot be projected onto the pillar P among the focused images. Can be reduced. For this reason, it is possible to display on the pillar P a natural and uncomfortable image having high continuity with the actual background seen through the windows W1 and W2.

  (2) In the above embodiment, the angle θc formed by the horizontal axis of the optical axis AX of the camera 6 is compared with the angle θd formed by the horizontal direction of the blind area of the blind spot area A, and the angle θc is greater than the angle θd. In addition, the photographing plane CP is set to a position passing through the right end point VP2. When the angle θc is smaller than the angle θd, the photographing plane CP is set to a position that passes through the left end point VP1. For this reason, since the imaging plane CP can always be set in front of the virtual plane VP, the surplus area generated when the imaging plane CP is set at a position farther from the head center position Dc than the virtual plane VP is obtained. Can be reduced.

In addition, you may change the said embodiment as follows.
In the above embodiment, the road marking reference point Pc is set based on the map drawing data 17. However, the white line recognition processing is performed on the image data IM acquired from the camera 6, and the road marking white line (or (Yellow line) may be detected. Then, the reference point Pc may be set at the end or center of the detected white line.

  In the above embodiment, the virtual plane VP is adjusted to the road marking such as the pedestrian crossing Z, but the object to match the position of the virtual plane VP may be changed as appropriate. For example, the virtual plane VP may be matched with a road surface installation such as a traffic light. Further, for example, when a vehicle C is equipped with a radar or the like as an obstacle detection means for measuring a relative distance to an obstacle ahead of the vehicle, and a target obstacle such as a pedestrian or a bicycle is detected, the virtual plane VP You may make it match with an obstacle. For example, a known image process such as feature detection is used to determine whether the obstacle is a pedestrian or a bicycle.

  -The data which show the position of road markings, such as a pedestrian crossing, may be acquired from the server which distributes road-to-vehicle communication or vehicle-to-vehicle communication, or those data, for example. Data indicating the position of an obstacle such as a pedestrian may also be received from an external device such as another vehicle.

In the above embodiment, the voice processor 24 may be omitted when voice guidance is not used.
In the above embodiment, the projector 4 projects an image onto the inner surface Pa of the pillar P. However, a display as a thin display unit is provided inside the pillar P, and output data is output from the image processor 20 to the display. OD may be output.

  In the above embodiment, the camera 6 is provided outside the front pillar P and images the area blocked by the front pillar P. However, the camera 6 may be provided in other pillars such as the rear or side of the vehicle. For example, the camera 6 may be attached to the outside of the rear pillar at the rear of the vehicle, and the area blocked by the rear pillar by the camera 6 may be photographed. In this case, the projector 4 is provided at a position where an image can be projected on the inner surface of the rear pillar, and the image processor 20 projects an image of the blind spot area blocked by the rear pillar on the inner surface of the rear pillar based on the image data captured by the camera 6. . Accordingly, for example, when the vehicle C moves backward toward the parking frame, the driver D can confirm the parking frame blocked by the rear pillar in the projection image, so that the parking operation can be easily performed. Further, one camera 6 may be provided outside each of the plurality of pillars P, and a plurality of projectors 4 may be provided according to the position of the camera 6. Further, a plurality of cameras 6 may be provided on one pillar P.

  In the above embodiment, the projector 4 is provided inside the roof R of the vehicle C. However, the position of the projector 4 may be a position where an image can be projected on the inner surface of the pillar P, for example, above the dashboard (substantially You may make it provide in other positions, such as a center.

The block diagram of the driving assistance system of this embodiment. Explanatory drawing of the position detection sensor which detects the head of a driver. Explanatory drawing of the attachment position of a camera. Explanatory drawing of the setting position of a virtual plane. Explanatory drawing of a virtual plane and a camera imaging surface. Explanatory drawing of a virtual plane and a camera imaging surface. Explanatory drawing of a mask pattern. The side view of the vehicle explaining the position of a projector. The front view explaining the pillar inner surface where an image is projected. Explanatory drawing explaining the projection direction of a projector. Explanatory drawing of the process sequence of this embodiment. Explanatory drawing of the conventional virtual plane position. Explanatory drawing of the conventional camera imaging surface.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Driving assistance system, 2 ... Driving assistance unit as driving assistance device, 6 ... Camera as imaging device, 20 ... Virtual plane setting means, imaging surface setting means, image processing means, and image processor as output control means, 109 ... extra area, A ... blind spot area, AX ... optical axis, B1 ... non-display area, C ... vehicle, CP ... imaging plane, D ... driver, IM ... image data, La ... center line as a straight line, P ... pillar, VP ... virtual plane, VP1, VP2 ... end points.

Claims (4)

In a driving support method for photographing a blind spot area caused by a pillar of the vehicle using an imaging device attached to the vehicle and displaying an image corresponding to the blind spot area on the inside of the pillar.
While setting a virtual plane to project the image data input from the imaging device, the imaging surface of the imaging device is set to a position that passes through the end points of the virtual plane defined by the blind spot area generated by the pillar,
The image data focused on the imaging surface is acquired, the image data is coordinate-converted to the virtual plane, and the pillar is arranged inside the pillar based on the image data coordinate-converted to the virtual plane. A driving support method characterized by displaying an image corresponding to a blind spot area that occurs. In a driving support device that uses a photographing device attached to a vehicle to photograph a blind spot area caused by a pillar of the vehicle and displays an image photographed by the photographing device on the inside of the pillar.
Virtual plane setting means for setting a virtual plane for projecting image data input from the imaging device;
A photographing surface setting means for setting a photographing surface of the photographing device to a position passing through an end point of the virtual plane defined by a blind spot region generated by the pillar;
Image processing means for acquiring the image data focused on the imaging surface, and coordinate-converting the image data into the virtual plane;
A driving support apparatus, comprising: output control means for displaying an image of a blind spot area generated by the pillar on the inner side of the pillar based on the image data coordinate-converted into the virtual plane. In the driving assistance device according to claim 2,
The driving support device, wherein the shooting plane setting means sets a shooting plane of the shooting device at a position closer to the vehicle than the virtual plane. In the driving assistance device according to claim 2 or 3,
The virtual plane setting means acquires the position of a drawing reference object on a road surface around the vehicle, sets the virtual plane at a position passing through the position of the drawing reference object,
The imaging plane setting means includes the virtual plane on which the imaging planes intersect based on the direction of the optical axis of the imaging apparatus and the direction of a straight line connecting the head position of the driver of the vehicle and the position of the drawing reference object. A driving support device characterized by selecting an end point.

Images