JP5067169B2 - Vehicle parking assistance apparatus and image display method - Google Patents

Description

  The present invention relates to a vehicle parking assistance apparatus and an image display method for displaying an image around a host vehicle on a display unit and assisting a driving operation during parking of a driver of the host vehicle.

Conventionally, a virtual vehicle such as computer graphics that generates a bird's-eye view of the surroundings of the vehicle from a virtual viewpoint above the vehicle is generated using images taken by a plurality of in-vehicle cameras, and the vehicle is displayed on the bird's-eye view image. A technique is known in which images are superimposed and displayed on display means in a vehicle interior. In addition, as a technique for displaying a bird's-eye view image of this type and assisting the driving operation of the driver at the time of parking, Patent Document 1 discloses an image displayed on the display means when the vehicle enters a parking posture. From an image in a normal display form in which a bird's-eye view image around the virtual vehicle image changes as the vehicle moves, to an image in a display form in which the virtual vehicle image moves on the bird's-eye view image as the vehicle moves The technology of switching is described.
JP 2007-183877 A

  As a display form of the image displayed on the display means at the time of parking, in order to objectively image the movement of the own vehicle at the time of parking, the overhead image is fixed and the virtual own vehicle image is an overhead image as the own vehicle moves. It is considered that the display mode moving above is more advantageous because it can objectively grasp the positional relationship between the vehicle position and the parking target position. On the other hand, in a scene where the steering operation is actually performed, the display mode in which the virtual vehicle image is fixed and the overhead image around the virtual vehicle image changes as the vehicle moves is the reference of the vehicle. In addition, the steering operation direction and the like can be determined intuitively, which is advantageous. However, in the technique described in Patent Document 1, when the host vehicle enters a parking posture, only the display image in which the virtual host vehicle image moves on the overhead image as the host vehicle moves is displayed on the display unit. For this reason, it is assumed that the driver cannot instantaneously determine the direction of operation, particularly during steering operation.

  The present invention was devised in view of the conventional situation as described above, and is a vehicle parking assistance device and image display that can more effectively support the driving operation of the driver of the own vehicle during parking. It aims to provide a method.

In order to achieve the above object, the present invention superimposes a virtual vehicle image on the center of a bird's-eye view image generated at the present time, and displays the bird's-eye view around the virtual vehicle image changing as the vehicle moves. The first display image of the form is generated, and the virtual vehicle image is superimposed on the vehicle position on the overhead view image generated at the time when the parking target position is set, and the virtual display is performed on the overhead image as the vehicle moves. A second display image in a display form in which the host vehicle image moves is generated, and the first display image and the second display image are displayed side by side on the display means.
In the present invention, the area around the host vehicle displayed in the second display image is wider than the area around the host vehicle displayed in the first display image. A frame indicating the parking target position by superimposing and displaying on the second display image at least one of the frame graphic indicating the target position and the frame graphic indicating the parking start position suitable for starting the parking operation at the parking target position. Neither the figure nor the frame figure indicating the parking start position is superimposed on the first display image.

  According to the present invention, the first display image in a display form in which a bird's-eye view image around the virtual vehicle image changes as the vehicle moves, and the display in which the virtual vehicle image moves on the bird's-eye image as the vehicle moves. Since the second display image of the form is displayed side by side on the display means, it is possible to more effectively support the driving operation when the driver of the own vehicle is parked.

  Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing a schematic configuration of a vehicle parking assistance apparatus to which the present invention is applied. The vehicle parking assistance apparatus according to the present embodiment uses the image processing unit 10 to perform image processing on images captured by four vehicle-mounted cameras, that is, the front camera 1, the rear camera 2, the right side camera 3, and the left side camera 4 mounted on the host vehicle. The display image is generated, and this display image is displayed on the display device 5 installed in the passenger compartment of the host vehicle. Four in-vehicle cameras 1 to 4 are connected to the input side of the image processing unit 10, The display device 5 is connected to the output side of the image processing unit 10. The image processing unit 10 is connected to an operation input device 6 that receives an operation input from the driver of the host vehicle and various vehicle sensors 7 that detect the behavior of the host vehicle.

  The four in-vehicle cameras including the front camera 1, the rear camera 2, the right side camera 3, and the left side camera 4 are, for example, wide-angle CCD cameras or CMOS cameras having an angle of view of about 180 degrees. It is attached at a suitable location around the vehicle so that images of all areas surrounding the vehicle can be photographed at -4. Specifically, as shown in FIG. 2, for example, the front camera 1 is installed on a front grill of the own vehicle and shoots an image of a region A1 ahead of the own vehicle in a direction looking down obliquely with respect to the ground. The rear camera 2 is installed in a roof spoiler or the like of the own vehicle and shoots an image of the area A2 behind the own vehicle in a direction looking down obliquely with respect to the ground. The right side camera 3 is installed on a right side mirror of the host vehicle and shoots an image of the area A3 on the right side of the host vehicle in a direction looking down obliquely with respect to the ground. Further, the left side camera 4 is installed on a left side mirror of the own vehicle and shoots an image of a region A4 on the left side of the own vehicle in a direction looking down obliquely with respect to the ground. Signals of images taken by these four in-vehicle cameras 1 to 4 are sent to the image processing unit 10 as needed.

  The display device 5 displays a display image generated by the image processing unit 10, and for example, a liquid crystal display is used. The display device 5 is installed at a position that is easy to see for the driver of the host vehicle, such as a center console in the cabin of the host vehicle, and the display generated by the image processing unit 10 based on the image signal from the image processing unit 10. Display an image.

  The operation input device 6 receives an operation input by the driver of the own vehicle, and various operation buttons, a joystick, etc. are used, for example. Further, a touch panel may be provided on the display screen of the display device 5, and this touch panel may be used as the operation input device 6. An operation input by the driver of the own vehicle using the operation input device 6 is sent to the image processing unit 10.

  The in-vehicle sensor 7 detects the behavior of the host vehicle, and for example, a vehicle speed sensor, a steering angle sensor, or a shift signal sensor is used. A vehicle speed sensor detects the vehicle speed of the own vehicle. The steering angle sensor detects the steering angle of the host vehicle. The shift signal sensor detects the shift position (gear position) of the host vehicle. Sensor signals representing the behavior of the vehicle detected by these onboard sensors 7 are sent to the image processing unit 10 as needed.

  The image processing unit 10 is configured to include an information processing device such as a microcomputer in which a program for processing an image is stored, and the in-vehicle cameras 1 to 4 are photographed by executing this program with an arithmetic device such as a CPU. A display image to be displayed on the display device 5 is generated using the image. When the configuration of the image processing unit 10 is viewed functionally, as shown in FIG. 3, as shown in FIG. 3, the image input unit 11, the overhead image generation unit 12, the parking target position setting unit 13, and the overhead image storage unit 14. And a vehicle movement amount calculation unit 15, a first display image generation unit 16, a second display image generation unit 17, and an image output unit 18.

  The image input unit 11 inputs images captured by four vehicle-mounted cameras, that is, the front camera 1, the rear camera 2, the right side camera 3, and the left side camera 4.

  The bird's-eye view image generation unit 12 uses the captured images of the in-vehicle cameras 1 to 4 input by the image input unit 11 to generate a bird's-eye view image looking down from the virtual viewpoint above the vehicle. FIG. 4 is a diagram for explaining the outline of the processing for generating the overhead image. The image 100 in FIG. 4 is an overhead image generated by the overhead image generation unit 12, the image 101 is a captured image of the front camera 1, and the image Reference numeral 102 denotes a captured image of the rear camera 2, an image 103 is a captured image of the right side camera 3, and an image 104 is a captured image of the left side camera 4. In the example of FIG. 4, in order to illustrate the state of overhead image generation in an easy-to-understand manner, each on-vehicle camera 1 to 4 captures an image including a reference line drawn on the ground around the vehicle. ing.

  The bird's-eye view image generation unit 12 performs a coordinate conversion process on the captured image 101 of the front camera 1 according to a known viewpoint conversion algorithm so that the area ahead of the host vehicle (the area A1 in FIG. 2) is located above the host vehicle. An image 100a looking down from the virtual viewpoint is generated. Further, the bird's-eye view image generation unit 12 performs a similar coordinate conversion process on the captured image 102 of the rear camera 2 so that the area behind the own vehicle (area A2 in FIG. 2) is viewed from the virtual viewpoint above the own vehicle. A look-down image 100b is generated. Further, the bird's-eye view image generation unit 12 performs the same coordinate conversion processing on the captured image 103 of the right side camera 3 and the captured image 104 of the left side camera 4, and thereby the area on the right side of the vehicle (in FIG. An image 100c in which the area A3) is looked down from the virtual viewpoint above the own vehicle, and an image 100d in which the area on the left side of the own vehicle (area A4 in FIG. 2) is looked down from the virtual viewpoint is generated. And the bird's-eye view image production | generation part 12 connects these images 100a-100d, and produces | generates the bird's-eye view image 100 which looked down the surroundings of the own vehicle from the virtual viewpoint above the own vehicle. The center of the bird's-eye view image 100 is the position where the vehicle is present when the bird's-eye view image 100 is generated.

  The generation of the bird's-eye view image by the bird's-eye view image generation unit 12 is repeatedly performed each time a new captured image is input to the image input unit 11 from each of the in-vehicle cameras 1 to 4. Then, the overhead image generated by the overhead image generation unit 12 is supplied to the first display image generation unit 16 as needed. Further, when the parking target position of the host vehicle is set by the parking target position setting unit 13, the overhead image generated by the overhead image generation unit 12 at that time is stored in the overhead image storage unit 14.

  The parking target position setting unit 13 sets, for example, a parking target position that becomes a parking target of the own vehicle in accordance with an operation input of the driver of the own vehicle using the operation input device 6. When the parking target position of the own vehicle is set by the parking target position setting unit 13, information to that effect is supplied to the overhead image generation unit 12, the own vehicle movement amount calculation unit 15, and the second display image generation unit 17. Is done.

  The overhead image storage unit 14 stores the overhead image generated by the overhead image generation unit 12 when the parking target position of the host vehicle is set by the parking target position setting unit 13. That is, when the bird's-eye view image generation unit 12 receives information from the parking target position setting unit 13 that the parking target position of the own vehicle has been set, the bird's-eye view image generated at that time is stored in the bird's-eye view image storage unit 14.

  The own vehicle movement amount calculation unit 15 receives sensor signals from various in-vehicle sensors 7, specifically, a vehicle speed signal from the vehicle speed sensor indicating the vehicle speed of the own vehicle, and a steering angle sensor indicating the steering steering angle of the own vehicle. Based on the steering angle signal of the vehicle and the shift position signal from the shift signal sensor indicating the shift position of the vehicle, the vehicle until the parking target position of the vehicle is set by the parking target position setting unit 13 until the present time. The movement direction and amount of movement are calculated. Information about the moving direction and the moving amount of the vehicle calculated by the moving amount calculating unit 15 is supplied to the second display image generating unit 17 as needed.

  The first display image generation unit 16 fixes the virtual vehicle image representing the vehicle as a still image at the center of the overhead image generated by the overhead image generation unit 12 and moves the virtual vehicle image along with the movement of the vehicle. A first display image (hereinafter referred to as a real-time image) in a display form in which the surrounding overhead image changes is generated. In other words, when the overhead image generated by the overhead image generation unit 12 is supplied at the present time, the first display image generation unit 16 has a virtual own vehicle image such as computer graphics representing the vehicle at the center of the overhead image. Is superimposed. This process is performed every time a new overhead image is generated by the overhead image generation unit 12. When the host vehicle moves, the bird's-eye view image generated by the bird's-eye view image generation unit 12 changes. Therefore, by superimposing the virtual host vehicle image on the center of the bird's-eye view image, A real-time image having a display form in which the overhead image changes is generated. The real-time image generated by the first display image generation unit 16 is supplied to the image output unit 18 as needed.

  The second display image generation unit 17 fixes the overhead image generated by the overhead image generation unit 12 at the time when the parking target position of the host vehicle is set by the parking target position setting unit 13 as a still image. Then, a second display image (hereinafter referred to as an objective image) in a display form in which the virtual own vehicle image representing the own vehicle moves in accordance with the actual movement of the own vehicle is generated. That is, the second display image generation unit 17 sets the parking target position of the own vehicle by the parking target position setting unit 13, and stores the overhead image generated by the overhead image generation unit 12 at that time in the overhead image storage unit 14. Then, based on the information on the moving direction and the moving amount of the own vehicle calculated by the own vehicle moving amount calculating unit 15, the own vehicle position on the overhead image stored in the overhead image storage unit 14 is specified. This process is repeated whenever the moving direction and the moving amount of the own vehicle calculated by the own vehicle moving amount calculation unit 15 change due to the movement of the own vehicle. Then, the second display image generation unit 17 superimposes a virtual vehicle image such as computer graphics representing the vehicle on the vehicle position on the identified bird's-eye view image, and on the bird's-eye view image as the vehicle moves. An objective view image in a display form in which the virtual vehicle image moves is generated. The objective image generated by the second display image generation unit 17 is supplied to the image output unit 18 as needed.

  When the parking target position of the own vehicle is not set and only the real-time image generated by the first display image generation unit 16 is supplied, the image output unit 18 displays the image signal of the real-time image as a display device. 5 is output. Thereafter, the parking target position of the host vehicle is set by the parking target position setting unit 13, and the real-time image generated by the first display image generation unit 16 and the objective image generated by the second display image generation unit 17 are generated. When each is supplied, the real-time image and the objective image are arranged into one display image, and an image signal of the display image is output to the display device 5 as needed. As a result, the display device 5 displays only a real-time image in a display form in which the overhead image around the virtual vehicle image changes as the vehicle moves before the parking target position of the vehicle is set. When the parking target position of the host vehicle is set, a real-time image in a display form in which the overhead view image around the virtual host vehicle image changes as the host vehicle moves and a virtual image on the overhead view image as the host vehicle moves are displayed. The objective view image of the display form in which the own vehicle image moves is arranged and displayed at the same time.

  Here, a specific example of a display image displayed on the display device 5 will be described in more detail with reference to FIGS. 5 to 9 assuming a scene in which the host vehicle performs parallel parking.

  While the target parking position of the host vehicle is not set, as shown in FIG. 5, only the real-time image P1 in which the virtual host vehicle image V is superimposed on the center of the overhead view image representing the current state around the host vehicle is displayed. 5 is displayed. In the real-time image P1 displayed on the display device 5, when the host vehicle moves, the bird's-eye view image around the virtual host vehicle image V changes, and the current state around the host vehicle is always displayed.

  Thereafter, the driver of the host vehicle finds a space where parking is possible while viewing the real-time image P1 displayed on the display device 5, for example, and uses the operation input device 6 to set the parking space as the parking target position. When the operation input is performed, the position designated by the driver according to the operation input by the driver is set as the parking target position. When the parking target position of the host vehicle is set, the display device 5 has a real-time image in which the virtual host vehicle image V is superimposed on the center of the overhead view image representing the current state around the host vehicle, as shown in FIG. The image P1 and the objective image P2 in which the virtual vehicle image V is superimposed on the current position of the host vehicle on the bird's-eye view image showing the surroundings of the host vehicle when the parking target position is set are displayed side by side. The display image illustrated in FIG. 6 is an image displayed on the display device 5 immediately after the target parking position of the host vehicle is set. Therefore, the virtual host vehicle image V is an overhead image in the objective image P2. It is superimposed on the center.

  It is desirable that the timing for setting the parking target position, that is, the timing for storing the bird's-eye view image, is a timing at which a three-dimensional object does not exist between the parking target position to be set and the host vehicle. This is a bird's-eye view image generated by converting the captured images of the in-vehicle cameras 1 to 4, and the three-dimensional object falls in a direction away from the own vehicle, and the parking target position to be set and the own vehicle If a parking target position is set when a three-dimensional object is present between the two objects, an objective image P2 is generated using an overhead image with the parking target position hidden by a three-dimensional object that has collapsed. This is because it becomes an obstacle for the driver of the own vehicle to objectively determine the positional relationship between the own vehicle and the parking target position by viewing the objective image P2.

  Further, when the real-time image P1 and the objective image P2 are displayed side by side on the display device 5, as shown in FIG. 6, the area around the vehicle displayed by the objective image P2 is displayed as the real-time image P1. When the image area of the area around the vehicle displayed as a bird's-eye view image is compared, the objective image P2 is wider than the real-time image P1. It is desirable. In order to realize this, for example, an overhead image including a wide area around the vehicle is generated when the overhead image is generated, and the wide overhead image is used as it is when generating the objective image P2. When generating the real-time image P1, the vicinity of the center of a wide range of bird's-eye view images may be cut out, and the cut-out bird's-eye view image may be enlarged and used in accordance with the display size. In this way, if the area around the host vehicle displayed in the objective image P2 is wider than the area around the host vehicle displayed in the real-time image P1, the objective image P2 has an overhead image. The displayable range of the virtual vehicle image V can be increased, and the state in the vicinity of the vehicle can be confirmed in more detail in the real-time image P1.

  When the vehicle moves after the parking target position of the vehicle is set, the display image displayed on the display device 5 is the image shown in FIG. 7, the image shown in FIG. 8, the image shown in FIG. 8, and the image shown in FIG. Transition to an image. The image shown in FIG. 7 is an example of a display image displayed on the display device 5 when the vehicle moves forward from a position where the parking target position of the vehicle is set to a position where parking starts. The image is an example of a display image displayed on the display device 5 when the host vehicle is traveling backward to the parking target position, and FIG. 9 is displayed on the display device 5 when the host vehicle reaches the parking target position. It is an example of a displayed image. As can be seen from the transition of the images shown in FIGS. 6 to 9, after the parking target position of the host vehicle is set, the display device 5 has an overhead view around the virtual host vehicle image V as the host vehicle moves. A real-time image P1 with a display form in which the image changes and an objective image P2 with a display form in which the virtual vehicle image V moves on the overhead image as the host vehicle moves are displayed side by side.

  Next, an example of processing by the image processing unit 10 of the vehicle parking assistance apparatus of the present embodiment will be described with reference to FIG. FIG. 10 illustrates a predetermined period (for example, the in-vehicle camera 1) by the image processing unit 10 from when the vehicle parking support device is activated by pressing the parking button by the driver of the own vehicle until the parking operation of the own vehicle is completed. It is a flowchart which shows the flow of a series of processes repeatedly performed for every (-4 imaging | photography period).

  When the flow of FIG. 10 is started, the image processing unit 10 first inputs captured images of the in-vehicle cameras 1 to 4 through the image input unit 11 in step S1, and the overhead image generation unit 12 performs step in step S2. Using the captured images of the in-vehicle cameras 1 to 4 input in S <b> 1, a bird's-eye view image in which the surroundings of the vehicle is looked down from a virtual viewpoint above the vehicle is generated. In step S3, the first display image generation unit 16 generates a real-time image by superimposing the virtual vehicle image representing the vehicle on the center of the overhead image generated in step S2.

  Next, in step S4, it is determined whether the parking target position of the host vehicle has already been set. If the parking target position of the host vehicle has not been set, the host vehicle for setting the parking target position in step S5. It is determined whether the operation input by the driver is made. Here, if an operation input for setting the parking target position is not made, in step S6, the image output unit 18 outputs the image signal of the real-time image generated in step S3 to the display device 5 and displays it. Only the real-time image is displayed on the device 5 and the process returns.

  On the other hand, if it is determined in step S5 that an operation input for setting the parking target position has been made, in step S7, the parking target position setting unit 13 causes the vehicle's own vehicle according to the operation input by the driver of the own vehicle. The parking target position is set and the process proceeds to step S8. At this time, the overhead image generated in step S <b> 2 is stored in the overhead image storage unit 14.

  If the parking target position of the own vehicle has already been set (YES in step S4), or if a new parking target position has been set in step S7, then in step S8, the amount of movement of the own vehicle is calculated. The unit 15 calculates the moving direction and the moving amount of the host vehicle from when the parking target position of the host vehicle is set until the present time. In step S9, the second display image generating unit 17 calculates the moving direction and the moving amount. Based on the moving direction and moving amount of the host vehicle, the position of the host vehicle on the bird's-eye view image stored in the bird's-eye view image storage unit 14 is specified, and a virtual host vehicle image representing the host vehicle is superimposed on the position to objectively A visual image is generated. In step S10, the image output unit 18 outputs an image signal of a display image in which the real-time image generated in step S3 and the objective image generated in step S10 are arranged to the display device 5, and the display image 5 Display the real-time image and the objective image side by side and return.

  The series of processes described above are repeated for each predetermined period (for example, the imaging period of the in-vehicle cameras 1 to 4) until the parking operation of the own vehicle is finished, and when the parking operation of the own vehicle is finished, FIG. This flow ends. The end of the parking operation of the host vehicle can be determined, for example, by detecting that the shift position of the host vehicle is parked by a shift position sensor.

  As described above in detail with specific examples, the vehicular parking support apparatus according to the present embodiment, when the parking target position of the own vehicle is set, the virtual vehicle image of the vehicle is moved along with the movement of the own vehicle. A real-time image in a display form in which a surrounding bird's-eye view image changes and an objective image in a display form in which the virtual vehicle image moves on the bird's-eye view image as the vehicle moves are displayed side by side on the display device 5. Therefore, according to this vehicle parking assist device, the driver's driver can visually indicate the movement of the vehicle at the time of parking to the driver of the own vehicle by the objective image, and the driver's driver can change the steering operation direction by the real-time image. It is possible to intuitively grasp the driving operation when the driver of the own vehicle is parked.

  Moreover, according to the vehicle parking assistance apparatus of the present embodiment, when the real-time image and the objective image are displayed side by side on the display device 5, the objective image has a wider range around the vehicle than the real-time image. Since the area is displayed, the display range of the virtual vehicle image in the objective image can be enlarged, and the real-time image allows the driver of the vehicle to grasp the state in the vicinity of the vehicle in detail. Can do.

  Moreover, according to the vehicle parking assistance apparatus of this embodiment, since the real-time image is displayed on the display device 5 at the time before the parking target position of the own vehicle is set, the driver of the own vehicle While looking at the real-time image displayed on the display device 5, it is possible to search for a space where the vehicle can be parked. Furthermore, since the parking target position of the own vehicle is set according to the operation input of the driver of the own vehicle using the operation input device 6, the parking target position is simply set at a position desired by the driver of the own vehicle. be able to.

  In the example described above, the setting of the parking target position is performed according to the operation input by the driver of the own vehicle using the operation input device 6, but the parking target position is automatically set in the image processing unit 10. It is also possible to set. For example, when a sensor for detecting an obstacle around the vehicle such as an ultrasonic sonar, a laser radar device, or a millimeter wave radar device is attached to the vehicle, the parking target position setting unit 13 of the image processing unit 10 is used. However, the sensor information from the obstacle detection sensor is acquired, the parking space around the vehicle is detected based on the sensor information from the obstacle detection sensor, and the detected parking space is set as the parking target position. Can be set. In addition, the parking target position setting unit 13 of the image processing unit 10 can capture the captured images of the in-vehicle cameras 1 to 4 and perform analysis processing (for example, white line recognition by edge detection) to park around the own vehicle. The space may be detected, and the detected parking space may be set as the parking target position.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In this embodiment, a frame graphic indicating a parking target position, a frame graphic indicating a parking start position suitable for starting a parking operation to the parking target position, and an ideal course of the own vehicle from the parking start position to the parking target position By drawing various guidance information such as the predicted course when the current steering angle of the host vehicle is maintained on the objective image and displaying it on the display device 5, the driving operation by the driver of the host vehicle during parking can be performed. It is intended to support more effectively.

  FIG. 11 is a block diagram showing a functional configuration of the image processing unit 10 in the vehicle parking assistance apparatus of the present embodiment. In the vehicle parking assistance apparatus of the present embodiment, as shown in FIG. 11, a guidance information drawing unit 19 is added as a functional configuration of the image processing unit 10 in addition to the configuration described in the first embodiment. The guidance information drawing unit 19 draws various types of guidance information that serves as an index of a driving operation by the driver of the own vehicle during parking on the objective image generated by the second display image generation unit 17.

  Specifically, when the parking target position of the host vehicle is set by the parking target position setting unit 13, the guidance information drawing unit 19 sets the parking target position on the objective image generated by the second display image generation unit 17. That is, the parking target position on the overhead image stored in the overhead image storage unit 14 is grasped, and a frame figure (hereinafter referred to as a parking target frame) indicating the parking target position is drawn at this position (parking). Target position drawing means). In addition, the guidance information drawing unit 19 displays, for example, on the real-time image generated by the first display image generation unit 16 before the parking target position setting unit 13 sets the parking target position of the host vehicle. A frame figure corresponding to the parking target frame is drawn at a position designated as a candidate parking target position by using the operation input device 6 or a position detected as a parking available space by the parking target position setting unit 13. The driver of the vehicle may recognize that this position is a candidate parking target position. At this time, for example, a message such as “Can this position be set as a parking target position?” Is displayed on the display device 5 or communicated by voice to set the parking target position for the driver of the vehicle. It is desirable to prompt the operation input for In addition, for example, by displaying a message such as “Move and rotate the displayed frame to the position where you want to park” on the display device 5 or by outputting it by voice, the driver of the vehicle will receive the desired parking target. The position may be selected. Then, when the driver of the own vehicle performs an operation input to set the position as the parking target position using the operation input device 6, and when the position is set as the parking target position by the parking target position setting unit 13, The guidance information drawing unit 19 draws a parking target frame at the parking target position on the objective image generated by the second display image generation unit 17.

  When the parking target position setting unit 13 sets the parking target position of the host vehicle, the guidance information drawing unit 19 monitors the steering angle signal from the steering angle sensor and grasps the current steering steering angle of the host vehicle. Then, based on the current steering steering angle of the host vehicle, a parking start position suitable for starting the parking operation at the parking target position is obtained. This parking start position is, for example, a position that can be reached while maintaining the current steering steering angle of the host vehicle, and if the steering operation is turned back at that position and the parking operation is started, the steering steering angle is set to the parking target position. It is a position that can be reached while keeping constant. And the guidance information drawing part 19 draws the frame figure (henceforth a parking start frame) which shows a parking start position in the parking start position on the objective image produced | generated by the 2nd display image production | generation part 17. Parking start position drawing means).

  Moreover, the guidance information drawing part 19 calculates | requires the ideal course of the own vehicle from this parking start position to a parking target position, if the own vehicle reaches | attains a parking start position. For example, the ideal course is a course that is reached from the parking start position to the parking target position while keeping the steering angle constant. Then, the guidance information drawing unit 19 draws the ideal course on the objective image generated by the second display image generation unit 17 (ideal course drawing means). Further, when the host vehicle reaches the parking start position, the guidance information drawing unit 19 monitors the steering angle signal from the steering angle sensor to grasp the current steering steering angle of the host vehicle, and the current steering steering of the host vehicle. A predicted course when the angle is maintained is obtained, and this predicted course is drawn on the objective image generated by the second display image generation unit 17 (predicted course drawing means). The ideal course of the vehicle is the course that reaches the target parking position from the parking start position with the steering steering angle constant, and the predicted course of the vehicle is the course when the current steering steering angle of the vehicle is maintained. If the driver of the own vehicle turns the steering to the optimum steering angle when the own vehicle reaches the parking start position, the predicted course on the objective image overlaps the ideal course.

  In the present embodiment, when the parking target position of the host vehicle is set by the parking target position setting unit 13 and the objective image is generated by the second display image generating unit 17, the guidance information drawing unit is displayed on the objective image. Various guide information as described above is drawn by 19. Then, the real-time image generated by the first display image generation unit 16 and the objective image generated by the second display image generation unit 17 and on which various types of guide information are drawn by the guide information drawing unit 19 are arranged. An image signal of the display image is output from the image output unit 18 to the display device 5 as needed. As a result, the display device 5 displays the real-time image and the objective image on which various types of guide information are drawn side by side.

  Here, assuming the parallel parking scene exemplified in the first embodiment, FIGS. 12 to 15 are shown as specific examples of display images displayed on the display device 5 in the vehicle parking assistance device of the present embodiment. Further details will be described with reference to FIG.

  In this embodiment, when the parking target position of the own vehicle is set, the display device 5 shows the real-time image P1, the parking target frame F1 indicating the parking target position, and the parking start position, as shown in FIG. The objective image P2 on which the parking start frame F2 is drawn is displayed side by side. The parking target frame F1 and the parking start frame F2 on the objective image P2 are displayed in different display colors so that they can be clearly distinguished, or only the parking target frame F1 is colored in the frame. It is desirable that the display forms differ from each other. In addition, when the parking target position of the own vehicle is set and a display image as shown in FIG. 12 is displayed on the display device 5, for example, “the vehicle is placed in the displayed red frame (parking start frame F2)”. It is desirable to guide the host vehicle to the parking start position by displaying a message such as “Please move” along with the display device 5 or by outputting it by voice.

  Thereafter, when the host vehicle moves toward the parking start position, the virtual host vehicle image V on the objective image P2 moves with the movement of the host vehicle, and when the host vehicle reaches the parking start position, it is shown in FIG. In this way, the parking start frame F2 drawn on the objective image P2 and the virtual vehicle image V overlap. At this time, for example, by highlighting the parking start frame F2 on the objective image P2, the driver of the own vehicle recognizes that the own vehicle has reached the parking start position, that is, needs to move to the parking operation. It is desirable to make it. In addition, for example, a message such as “A parking start position has been reached. Please put the shift lever in the reverse direction” is displayed on the display device 5 or output by voice so as to prompt the start of the parking operation. May be.

  In addition, when the start of the parking operation is detected, for example, when the shift position sensor detects that the shift position of the host vehicle has been reversed, it corresponds to the ideal course to the parking target position and the current steering steering angle. As shown in FIG. 14, the predicted course is calculated, and an objective image P2 in which the ideal course L1 and the predicted course L2 are drawn in addition to the parking target position F1 is displayed on the display device 5 side by side with the real-time image P1. Is displayed. At this time, for example, "Turn the steering wheel while stopped and align the orange broken line (predicted course L2) with the pink broken line (ideal course L1). Then, move backward with the handle fixed. It is desirable to guide the host vehicle from the parking start position to the parking target position by displaying a message such as “.” Along with the display device 5 or outputting it by voice. The drawing of the parking start frame F2 on the objective image P2 ends when the own vehicle starts the parking operation.

  Thereafter, when the host vehicle moves from the parking start position toward the parking target position, the virtual host vehicle image V on the objective image P2 moves as the host vehicle moves, and when the host vehicle reaches the parking start position, As shown in FIG. 15, the parking target frame F1 drawn on the objective image P2 and the virtual vehicle image V overlap. Thereby, the driver of the own vehicle can recognize that parking has been completed. Further, for example, a message such as “Parking completed. Thank you very much.” May be displayed on the display device 5 or output by voice so that the driver of the own vehicle recognizes the completion of the parking operation. .

  Next, an example of processing by the image processing unit 10 of the vehicle parking assistance apparatus of the present embodiment will be described with reference to FIG. FIG. 16 illustrates a predetermined period (for example, the in-vehicle camera 1) by the image processing unit 10 from when the vehicle parking support apparatus is activated by pressing the parking button by the driver of the own vehicle until the parking operation of the own vehicle ends. It is a flowchart which shows the flow of a series of processes repeatedly performed for every (-4 imaging | photography period). Note that steps S11 to S19 in the flow of FIG. 16 are the same as steps S1 to S9 in the flow of FIG. 10 described in the first embodiment.

  That is, also in the vehicle parking assistance device of the present embodiment, the image processing device 10 displays the virtual vehicle image at the center of the overhead view image generated at the present time until the parking target position of the vehicle is set. Only the superimposed real-time image is displayed on the display device 5 (steps S11 to S16). Then, when the parking target position of the own vehicle is set, the bird's-eye view image generated at that time is stored (step S17), and thereafter, the moving direction and the moving amount of the own vehicle after the parking target position is set. Is calculated (step S18), and an objective image is generated by superimposing the virtual vehicle image on the own vehicle position on the overhead image generated when the parking target position is set (step S19).

  Thereafter, in this embodiment, the guidance information drawing unit 19 draws various types of guidance information on the objective image generated in step S19. That is, the guidance information drawing unit 19 first draws a parking target frame indicating the parking target position on the objective image generated in step S19 in step S20. If the vehicle is before the parking operation is started (NO in step S21), the guidance information drawing unit 19 obtains a parking start position suitable for starting the parking operation at the parking target position. The parking start frame indicating the parking start position is drawn on the objective image generated in step S19. On the other hand, if the own vehicle has reached the parking start position and the parking operation has started (YES in step S21), the guidance information drawing unit 19 continues until the own vehicle reaches the parking target position (step S23). NO), the ideal course of the host vehicle from the parking start position to the parking target position and the predicted course when the current steering steering angle of the host vehicle is maintained are obtained on the objective image generated in step S19. The ideal course and the predicted course of the host vehicle are drawn (step S24).

  In step S25, the image output unit 18 outputs an image signal of a display image in which the real-time image generated in step S3 and the objective image in which various types of guidance information are drawn are arranged to the display device 5, and displays them. A real-time image and an objective image on which various types of guidance information are drawn are displayed side by side on the image 5 and the process returns.

  The series of processes described above is repeated for each predetermined period (for example, the imaging period of the in-vehicle cameras 1 to 4) until the parking operation of the own vehicle is finished, and when the parking operation of the own vehicle is finished, FIG. This flow ends. The end of the parking operation of the host vehicle can be determined, for example, by detecting that the shift position of the host vehicle is parked by a shift position sensor.

  As described above in detail with reference to specific examples, according to the vehicle parking assistance device of the present embodiment, when the parking target position of the host vehicle is set, as in the first embodiment, Display real-time image in which the bird's-eye view around the virtual vehicle image changes as the vehicle moves, and an objective image in which the virtual vehicle image moves on the bird's-eye image as the vehicle moves Since the vehicle 5 is displayed side by side on the device 5, the driver's driver can visually indicate the movement of the vehicle at the time of parking by the objective image, and the driver's driver can determine the steering operation direction by the real-time image. It is possible to intuitively grasp the driving operation when the driver of the own vehicle is parked.

  Moreover, according to the vehicle parking assistance apparatus of the present embodiment, the parking target frame indicating the parking target position, the parking start frame indicating the parking start position suitable for starting the parking operation to the parking target position, and the parking start position Objectively view various guidance information that can be used as an indicator of driving operations by the driver of the vehicle, such as the ideal route of the vehicle from the vehicle to the parking target position, and the predicted route of the vehicle when the current steering steering angle of the vehicle is maintained. Since the image is drawn on the image and displayed on the display device 5, the driving operation of the driver of the own vehicle at the time of parking can be supported more finely.

  As described above, the first and second embodiments have been illustrated and described as specific examples of the vehicle parking assistance device to which the present invention is applied. However, each of the above embodiments is an application example of the present invention. The technical scope is not intended to be limited to the contents described in the above embodiments. That is, the technical scope of the present invention is not limited to the specific technical matters disclosed in the above embodiments, but includes various modifications, changes, alternative techniques, and the like that can be easily derived from this disclosure.

It is a block diagram which shows schematic structure of the driving assistance device for vehicles to which this invention is applied. It is a figure explaining the attachment position of four vehicle-mounted cameras, and the area | region which each vehicle-mounted camera image | photographs. It is a block diagram which shows the function structure of the image processing unit in the parking assistance apparatus for vehicles of 1st Embodiment. It is a figure explaining the outline | summary of the process which produces | generates a bird's-eye view image. It is a figure which shows an example of the image displayed on a display apparatus before the parking target position is set. It is a figure which shows an example of the image displayed on a display apparatus at the time of a parking target position being set. It is a figure which shows an example of the image displayed on a display apparatus, when the own vehicle moves after a parking target position is set. It is a figure which shows an example of the image displayed on a display apparatus, when the own vehicle is performing parking operation toward a parking target position. It is a figure which shows an example of the image displayed on a display apparatus when the own vehicle reaches | attains a parking target position. It is a flowchart which shows the flow of a series of processes repeatedly performed for every predetermined period by the image processing unit of the parking assistance apparatus for vehicles of 1st Embodiment. It is a block diagram which shows the function structure of the image processing unit in the parking assistance apparatus for vehicles of 2nd Embodiment. In the vehicle parking assistance apparatus of 2nd Embodiment, it is a figure which shows an example of the image displayed on a display apparatus when a parking target position is set. In the vehicle parking assistance apparatus of 2nd Embodiment, it is a figure which shows an example of the image displayed on a display apparatus when the own vehicle reaches | attains a parking start position. In the vehicle parking assistance apparatus of 2nd Embodiment, it is a figure which shows an example of the image displayed on a display apparatus when the own vehicle starts parking operation | movement. In the vehicle parking assistance apparatus of 2nd Embodiment, it is a figure which shows an example of the image displayed on a display apparatus when the own vehicle reaches | attains a parking target position. It is a flowchart which shows the flow of a series of processes repeatedly performed for every predetermined period by the image processing unit of the parking assistance apparatus for vehicles of 1st Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Front camera 2 Rear camera 3 Right side camera 4 Left side camera 5 Display apparatus 6 Operation input device 7 Car-mounted sensor 11 Image input part 10 Image processing unit 12 Overhead image generation part 13 Parking target position setting part 14 Overhead image storage part 15 Self Car movement amount calculation unit 16 First display image generation unit 17 Second display image generation unit 18 Image output unit 19 Guide information drawing unit

Claims (8)

A plurality of in-vehicle cameras attached around the vehicle,
An overhead image generation means for processing captured images of the plurality of in-vehicle cameras and generating an overhead image in which the surroundings of the vehicle are looked down from a virtual viewpoint above the vehicle;
Parking target position setting means for setting the parking target position of the own vehicle;
An overhead image storage means for storing the overhead image generated by the overhead image generation means when the parking target position is set;
Own vehicle movement amount calculation means for calculating the movement direction and movement amount of the own vehicle after the parking target position is set;
A display mode in which a virtual vehicle image representing the vehicle is superimposed at the center of the overhead image generated by the overhead image generation means at the present time, and the overhead image around the virtual vehicle image changes as the vehicle moves. First display image generating means for generating one display image;
Based on the direction and amount of movement of the own vehicle calculated by the own vehicle movement amount calculating means, the position of the own vehicle on the overhead image stored in the overhead image storage means is specified and stored in the overhead image storage means. A virtual vehicle image representing the vehicle is superimposed on the vehicle position on the overhead view image thus generated, and a second display image in a display form in which the virtual vehicle image moves on the overhead image as the vehicle moves is generated. Two display image generation means;
Display means for displaying side by side the first display image generated by the first display image generation means and the second display image generated by the second display image generation means;
Draw at least one of the frame graphic indicating the parking target position set by the parking target position setting means and the frame graphic indicating the parking start position suitable for starting the parking operation at the parking target position. Guidance information drawing means to perform,
Equipped with a,
The area around the host vehicle displayed in the second display image is wider than the area around the host vehicle displayed in the first display image,
None of the frame graphic indicating the parking target position and the frame graphic indicating the parking start position are superimposed on the first display image, and the frame graphic indicating the parking target position and the frame graphic indicating the parking start position; A vehicle parking assistance apparatus , wherein at least one of the frame figures is superimposed on the second display image . The display means, in the time before the parking target position is set, for a vehicle according to claim 1, characterized in that to display only the first display image generated by the first display image generating unit Parking assistance device. It further comprises an input means for accepting an operator's operation input,
The said parking target position setting means sets the said parking target position according to the operation input of the operator received with the said input means, The parking assistance apparatus for vehicles of Claim 1 or 2 characterized by the above-mentioned. The parking target position setting means detects a parking available space around the own vehicle based on sensor information from an obstacle detection sensor mounted on the own vehicle, and sets the detected parking available space as the parking target position. The parking assist device for a vehicle according to claim 1 or 2 , wherein The parking target position setting means detects an available parking space around the vehicle by analyzing an image taken by the in-vehicle camera, and sets the detected parking available space as the parking target position. Item 3. The vehicle parking assistance device according to Item 1 or 2 . An ideal route drawing unit for obtaining an ideal route of the host vehicle from the parking start position to the parking target position, and drawing the ideal route on the second display image generated by the second display image generation unit ;
Any of claims 3 to 5, characterized that you superimpose frame shapes of both frame graphic indicating the parking start position and a frame graphic indicating the target parking position on the second display image 1 The vehicle parking assistance apparatus according to Item . A predicted course drawing unit that obtains a predicted course of the host vehicle when the current steering steering angle of the host vehicle is maintained, and draws the predicted course on the second display image generated by the second display image generation unit; The vehicle parking assistance apparatus according to claim 6 , further comprising: The captured images of a plurality of in-vehicle cameras attached to the surroundings of the vehicle are processed to generate an overhead image in which the surroundings of the vehicle are looked down from a virtual viewpoint above the own vehicle, and the virtual vehicle representing the vehicle is displayed on the overhead image. An image display method in which a vehicle image is superimposed and displayed on a display means,
The virtual vehicle image is superimposed on the center of the overhead image generated at the present time, and a first display image in a display form in which the overhead image around the virtual vehicle image changes as the vehicle moves is generated. The virtual vehicle image is superimposed on the vehicle position on the overhead view image generated when the parking target position is set, and the virtual vehicle image moves on the overhead image as the vehicle moves. A second display image is generated, the first display image and the second display image are displayed side by side on the display means, and an area around the host vehicle displayed in the second display image is the first display. Parking area that is wider than the area around the vehicle displayed in the image, accepts the designation of the parking target position, and is suitable for starting the frame operation indicating the parking target position and the parking operation to the parking target position Less with a frame figure indicating the start position And characterized in that it also one of the frame shape is superimposed on the second display image, both the frame graphic indicating the parking start position and a frame graphic indicating the target parking position is not superimposed on the first display image Image display method.

Images