JP4910425B2 - Parking assistance device and parking assistance method - Google Patents

Description

  The present invention relates to a parking assistance device that converts an image captured by an in-vehicle camera into an image viewed from a virtual viewpoint above a vehicle, displays the image on a display screen of a display device, and assists a driver's driving operation when the vehicle is parked, and It relates to parking assistance methods.

  Conventionally, when a vehicle is parked, an image taken by an in-vehicle camera such as a rear camera installed at the rear of the vehicle is displayed on the display in the passenger compartment, and the vehicle occupant specifies the target parking space on the display screen of this display. Then, the parking assistance apparatus which assists steering operation etc. of a vehicle is proposed so that a vehicle can move to the target parking space. Also, in recent years, taking into account the fact that when the video taken by the in-vehicle camera is displayed on the display as it is, the white line indicating the parking space appears distorted, making it difficult to specify the target parking space. Attempts have also been made to convert a captured image into a bird's-eye view image viewed from a virtual viewpoint above the vehicle and display it on a display, and to perform an operation of designating a target parking space on the bird's-eye view image (for example, Patent Document 1) reference.).

In the parking assistance device described in Patent Document 1, a video image taken by an in-vehicle camera is converted into a bird's-eye view video and displayed on a touch-panel display screen, and a vehicle occupant is placed at an arbitrary position on the bird's-eye view video displayed on the display screen. When touched, the surrounding image is analyzed to specify the position and direction of the target parking space, or when the vehicle occupant performs an operation of tracing an arbitrary position on the bird's-eye view image, the position and the direction of tracing are determined. The position and direction of the target parking space are specified, and a frame figure having a size corresponding to the outer shape of the vehicle is superimposed and displayed in accordance with the specified position and direction of the target parking space. Then, the steering operation of the vehicle is assisted so that the vehicle can appropriately move to the target parking space on which the frame graphic is superimposed and displayed.
JP 2001-339716 A

  However, in the conventional parking assistance device described in Patent Document 1, the position and direction of the target parking space is specified on the overhead image according to the operation by the vehicle occupant, and the position and direction of the specified target parking space are specified. Since the frame figure of a size corresponding to the outer shape of the vehicle is superimposed and displayed, the parking space is identified until the target parking space is specified and the frame figure is superimposed and displayed at that position. It is difficult to determine whether the vehicle has a space large enough to park the vehicle, and when the vehicle occupant performs an operation to specify the position and direction of the target parking space on the overhead view image, an intuitive operation is performed. There is a problem that it is difficult and inferior in operability.

  The present invention was devised in view of the conventional situation as described above, and allows the vehicle occupant to intuitively specify the position and direction of the target parking space to improve operability. An object of the present invention is to provide a parking assistance device and a parking assistance method.

In order to achieve the above object, the present invention provides a display screen of a display device that displays a bird's-eye view video obtained by changing the viewpoint of a video image taken by an in-vehicle camera or a vehicle surrounding bird's-eye view video obtained by changing the viewpoint of a video image taken by a plurality of vehicle cameras. Based on the minimum turning radius of the vehicle, multiple recommended parking start positions that can be parked without switching back to the target parking space are calculated, and figures representing the multiple recommended parking start positions are displayed as a bird's-eye view around the vehicle. was so that is displayed to be superimposed on.

According to the present invention, in the vehicle both around bird's-eye view, the target parking space in the available parking more suggested parking start position without performing forward turning operation is displayed superimposed, and interfere with the trajectory of the vehicle to the target parking space near When such an obstacle exists, the selection of the parking start position is facilitated, and the operability of the vehicle is improved.

  Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
First, as a first embodiment of the present invention, for a parking assistance device that converts a captured image of a rear camera installed in the rear of a vehicle into a bird's-eye view image and displays it on a display screen of a display device installed in a vehicle interior. An example to which the present invention is applied will be described.

  FIG. 1 shows a schematic configuration of the parking assistance apparatus of the present embodiment. The parking assist apparatus shown in FIG. 1 includes an image processing unit 10 that creates a bird's-eye view image to be displayed as a main component, and a rear camera 1 is connected to the input side of the image processing unit 10, and the output side The display device 2 is connected. In addition to the rear camera 1, a display changeover switch 3, an operation input device 4, and a vehicle information acquisition device 5 are connected to the input side of the image processing unit 10.

  The rear camera 1 is a wide-angle camera having an angle of view of about 180 degrees, for example, and is installed at the rear of the vehicle. The rear camera 1 captures an image of a predetermined area behind the vehicle in response to a command from the image processing unit 10, converts the captured image into an electrical signal, and outputs, for example, a composite signal (an analog format video signal). ) To the image processing unit 10 via a predetermined analog communication line.

  The display device 2 is composed of, for example, a liquid crystal display installed in the passenger compartment, and displays a bird's-eye view image of the rear of the vehicle on the display screen by performing display processing based on the video signal output from the image processing unit 10. As the display device 2, a dedicated display device may be used for the parking assistance device of the present embodiment, but a display device used in other devices mounted on the vehicle such as an in-vehicle navigation device is used. You may do it. In this case, the video displayed on the display screen of the display device 2 is switched by operating the display changeover switch 3.

  The display changeover switch 3 is a switch operated by a vehicle occupant when displaying a bird's-eye view image of the rear of the vehicle on the display device 2, and is installed, for example, on an instrument panel or the like in the vehicle interior. When the vehicle occupant performs an operation to display a bird's-eye view image of the rear of the vehicle on the display device 2 using the display changeover switch 3, the information is input to the image processing unit 10.

  The operation input device 4 is for a vehicle occupant to input various instructions other than the start of the overhead view video display. For example, various input devices commonly used such as a combination switch, a joystick, and a voice input device. Is available. When a touch panel display having a touch panel display screen is used as the display device 2, the touch panel of the display device 2 is also used as the operation input device 4. When the vehicle occupant performs various operations using the operation input device 2, an instruction corresponding to the operation is input to the image processing unit 10.

  The vehicle information acquisition device 5 acquires various vehicle information representing the current steering operation angle, shift position, etc. of the vehicle. Various types of vehicle information acquired by the vehicle information acquisition device 5 are appropriately input to the image processing unit 10 in response to a request from the image processing unit 10.

  The image processing unit 10 creates a bird's-eye view video using the video captured by the rear camera 2, and a frame figure representing the target parking space and a target trajectory representing the target trajectory from the current position of the vehicle to the target parking space in the overhead view video. A line, an expected trajectory line representing an expected trajectory from the current position of the vehicle, and the like are superimposed as necessary and output to the display device 2 as a bird's-eye view image to be displayed. For example, by various electric / electronic elements The image processing circuit is configured, a digital image processing circuit configured by a semiconductor chip, and the like.

  In particular, as shown in FIG. 2, the image processing unit 10 includes an input frame buffer 11 that temporarily holds a captured video input as a video signal from the rear camera 1 for each frame, as shown in FIG. An output frame buffer 12 that temporarily holds a bird's-eye view image to be displayed to be output to the display device 2 for each frame, and a rear camera image that is held in the input frame buffer 11 is converted into a bird's-eye view image and output frames. A viewpoint conversion processing unit 13 for writing to the buffer 12. Further, the image processing unit 10 includes a frame graphic display position determination unit 14 that determines the display position and direction of the frame graphic to be displayed on the display device 2 by being superimposed on the overhead image, and the bird's eye view held in the output frame buffer 12. A frame graphic drawing unit 15 for drawing a frame graphic on the video, a target parking space specifying unit 16 for specifying a target parking space as a target for parking the vehicle, and a target for calculating a target trajectory from the current position of the vehicle to the target parking space A trajectory calculation unit 17, a target trajectory line drawing unit 18 that draws a target trajectory line on the overhead view video held in the output frame buffer 12, an expected trajectory calculation unit 19 that calculates an expected trajectory from the current position of the vehicle, and an output frame buffer An expected trajectory line drawing unit 20 is provided for drawing an expected trajectory line on the bird's-eye view video held at 12.

  The video signal of the captured video output from the rear camera 1 is A / D converted and decoded into an RGB signal, and then input to the image processing unit 10 as frame data constituting a frame of the captured video of the rear camera 1. Stored in the frame buffer 11. Then, the viewpoint conversion processing unit 13 of the image processing unit 10 uses, for example, a conversion map that describes a correspondence relationship between an address for each pixel of the input frame buffer 11 and an address for each pixel of the output frame buffer 12, and the like. A process of converting the captured video of the rear camera 1 stored in the buffer 11 into a bird's-eye view video looking down from a virtual viewpoint above the vehicle is performed, and the converted video data of the bird's-eye view is stored in the output frame buffer 12.

  The video data (RGB signal) of the overhead view video stored in the output frame buffer 12 is encoded into a composite signal and D / A converted at a timing synchronized with the imaging cycle of the rear camera 1 and then from the image processing unit 10. And output to the display device 2 via a predetermined analog communication line. Then, when the display device 2 performs a display operation based on the video signal from the image processing unit 10, for example, a bird's-eye view video as shown in FIG. 3A is displayed on the display screen of the display device 2. become. Note that the bird's-eye view image shown in FIG. 3 (a) is obtained by subjecting the captured image shown in FIG. 3 (b) taken by the rear camera 1 to a viewpoint conversion process in a scene where the vehicle is parked in parallel in a predetermined parking space. It was created.

  Further, in the parking assistance device of the present embodiment, the frame graphic drawing unit 15 of the image processing unit 10 draws a frame graphic on the overhead view video held in the output frame buffer 12, so that the display screen of the display device 2 is displayed. In addition, a frame figure having a size corresponding to the outer shape of the vehicle is displayed superimposed on the overhead view video. When the target parking space that is the target for parking the vehicle is specified, the target trajectory line drawing unit 18 draws the target trajectory line on the overhead view image held in the output frame buffer 12, and the predicted trajectory line drawing unit 20 Is drawn on the bird's-eye view video held in the output frame buffer 12 so that the target track line and the predicted track line are superimposed on the bird's-eye view video on the display screen of the display device 2. Yes.

  The display position and direction of the frame graphic on the display screen of the display device 2 are determined by the frame graphic display position determination unit 14 of the image processing unit 10. The frame figure display position determination unit 14 is, for example, when the vehicle occupant operates the display changeover switch 3 or vehicle information acquired by the vehicle information acquisition device 5 (for example, the shift position is back and the vehicle speed is equal to or less than a predetermined value). When a certain trigger for instructing the start of display of a bird's-eye view image is detected, such as when it is determined that the vehicle has entered a parking posture based on the The display position and direction are determined as a predetermined reference position and direction. Then, the frame graphic drawing unit 15 draws the frame graphic on the overhead video held in the output frame buffer 12 based on the information on the display position and direction determined by the frame graphic display position determining unit 14. For example, as shown in FIG. 4 (a), a frame figure F having a size corresponding to the outer shape of the vehicle is superimposed on the overhead view image in a predetermined direction at a predetermined reference position on the display screen of the display device 2. Will be displayed.

  It should be noted that the reference position and direction for displaying the frame figure F at the start of the display of such an overhead image is, for example, a direction parallel to the vehicle at a rear position that is a predetermined distance away from the current position of the vehicle. It may be determined so that the display position and direction are always constant. For example, in a parking space that is frequently used, the position and orientation of the parking space with respect to the vehicle when the vehicle enters the parking posture are known in advance. The location information of the parking space and the information on the display position and direction of the frame figure F are stored in association with each other, and when the vehicle approaches the parking space and enters the parking posture, You may make it display the frame figure F by the memorize | stored display position and direction at the time of a display start.

  In the parking assistance device of the present embodiment, the vehicle occupant uses the operation input device 4 such as a combination switch, joystick, voice input device, touch panel, etc., for the display position and direction of the frame figure F on the display screen of the display device 2. Can be changed to an arbitrary position and direction. That is, when the vehicle occupant performs an operation of changing the display position and direction of the frame figure F on the display screen of the display device 2 using the operation input device 4 after the display of the overhead view video is started, the operation content is displayed. The information shown is input to the frame graphic display position determination unit 14 of the image processing unit 10. When the information indicating the operation content of the vehicle occupant is input from the operation input device 4, the frame graphic display position determination unit 14 displays the display position of the frame graphic F on the display screen of the display device 2 according to the operation content. Determine the direction. Then, the frame graphic drawing unit 15 draws the frame graphic F on the overhead image held in the output frame buffer 12 based on the information on the display position and direction determined by the frame graphic display position determining unit 14. As a result, the display position and direction of the frame figure F on the display screen of the display device 2 are changed. For example, as shown in FIG. A frame figure F having a size corresponding to the outer shape of the vehicle is displayed superimposed on the overhead view video.

  Here, the operation of changing the display position and direction of the frame graphic F by the vehicle occupant may be performed, for example, by rotating the frame graphic F superimposed on the display screen of the display device 2 on the display screen or on the display screen. This is done by translating in the vertical and horizontal directions. Specifically, when the display position and direction of the frame figure F illustrated in FIG. 4A are changed to the display position and direction illustrated in FIG. 4B, the vehicle occupant, for example, as illustrated in FIG. First, the frame figure F is rotated around the midpoint of the diagonal line (arrow A in FIG. 5), the rotated frame figure F is translated in the vertical direction (arrow B in FIG. 5), and then What is necessary is just to operate in the procedure of parallelly moving in the horizontal direction (arrow C in FIG. 5). Note that the order of the operation of rotating the frame figure F, the operation of moving the frame figure F in the vertical direction, and the operation of moving the frame figure F in the horizontal direction is arbitrary, and may be an order in which the vehicle occupant can easily operate.

  Further, in the parking assist device of the present embodiment, when an occupant of the vehicle inputs an instruction to confirm the display position and direction of the frame figure F on the display screen of the display device 2 using the operation input device 4, When the target parking space specifying unit 16 of the processing unit 10 has a target parking space in the direction in which the frame graphic F is displayed at the position where the frame graphic F is displayed on the display screen of the display device 2. Recognize and identify it as the target parking space.

  When the target parking space is specified by the target parking space specifying unit 16 of the image processing unit 10, the target trajectory calculating unit 17 can move from the current position of the vehicle to the target parking space, for example, with a constant steering operation angle. An ideal trajectory is calculated as the target trajectory. Then, the target trajectory line drawing unit 18 draws a line representing the target trajectory on the overhead image held in the output frame buffer 12 based on the target trajectory information calculated by the target trajectory calculation unit 17. Thus, for example, as shown in FIG. 6A, the target track line L1, which serves as an index for moving the vehicle from the current position to the target parking space, is superimposed on the overhead image on the display screen of the display device 2. Will be displayed.

  When the target parking space is specified by the target parking space specifying unit 16 of the image processing unit 10, the predicted trajectory calculating unit 19 uses the steering operation angle information and the shift position information acquired by the vehicle information acquisition device 5. Then, the current steering operation angle and the traveling direction of the vehicle are determined, and based on this, a trajectory when the vehicle is moved from the current position along the traveling direction at the steering operation angle is calculated as an expected trajectory. Then, the predicted trajectory line drawing unit 20 draws a line representing the predicted trajectory on the bird's-eye view video held in the output frame buffer 12 based on the information on the predicted trajectory calculated by the predicted trajectory calculating unit 19. Thus, for example, as shown in FIG. 6A, an expected trajectory line L2 representing a trajectory when the vehicle is moved from the current position as it is is displayed superimposed on the overhead image on the display screen of the display device 2. Will be.

  In the parking assistance device of the present embodiment, as described above, when the target parking space is specified on the overhead view image displayed on the display screen of the display device 2, the target track line L1 and the predicted track line L2 are displayed. By displaying on the screen, the driving operation by the vehicle driver is supported so that the vehicle can be easily parked in the target parking space. That is, the driver of the vehicle looks at the display screen of the display device 2 as shown in FIG. 6A, for example, so that the predicted track line L2 reflecting the current steering operation angle of the vehicle overlaps the target track line L1. If the steering of the vehicle is operated and an operation for gradually moving the vehicle backward is performed in a state where the overhead view image as shown in FIG. 6B is displayed on the display screen of the display device 2, the vehicle is moved to the target parking space. It can be parked properly.

  Here, in the parking assistance device of the present embodiment, the flow of a series of processing by the image processing unit 10 when displaying the overhead view video on the display screen of the display device 2 will be described with reference to the flowcharts of FIGS. 7 and 8. To do. 7 is a main routine that is repeatedly executed by the image processing unit 10 in synchronization with the imaging cycle of the rear camera 1, and the processing flow in FIG. 8 shows the details of the drawing process in step S13 in FIG. This subroutine is shown.

  The processing flow in FIG. 7 is, for example, when the display changeover switch 3 is operated by a vehicle occupant or when it is determined that the vehicle has entered a parking posture based on the vehicle information acquired by the vehicle information acquisition device 5. This is started when a trigger for instructing start of display of the overhead view video is detected.

  When the processing flow of FIG. 7 is started, first, in step S11, a captured image captured by the rear camera 1 is taken into the image processing apparatus 10 and stored in the input frame buffer 11 as image data for each frame. Next, in step S12, the viewpoint conversion processing unit 13 converts the captured image of the rear camera 1 stored in the input frame buffer 11 into a bird's-eye view image looking down from the virtual viewpoint above the vehicle using a conversion map or the like. Processing is performed, and the converted video data of the overhead view video is stored in the output frame buffer 12.

  Next, in step S <b> 13, the frame graphic drawing unit 15 performs a process of drawing the frame graphic F on the overhead view video held in the output frame buffer 12, and the target trajectory line drawing unit 18 performs the target trajectory as necessary. The line L1 and the predicted trajectory line drawing unit 20 perform processing for drawing the expected trajectory line L2. In step S <b> 14, the video data of the overhead video to be displayed held in the output frame buffer 12 is output from the image processing device 10 to the display device 2. Thereby, the bird's-eye view image behind the vehicle on which the frame figure F, the target track line L1, the predicted track line L2, and the like are superimposed is displayed on the display screen of the display device 2.

  The processing flow of FIG. 7 is performed until the rear camera 1 is parked until the parking of the vehicle is completed and a trigger for instructing to stop the display of the overhead view image is detected or the power of the parking assist device is turned off. The process is repeated in synchronization with the shooting cycle, and the process ends when an instruction to stop the display of the overhead view image or the power-off of the parking support apparatus is performed.

  Drawing processing of the frame figure F, the target trajectory line L1, and the predicted trajectory line L2 in step S13 of FIG. 7 is performed according to the processing flow of FIG. When the processing flow of FIG. 8 is started, first, in step S101, the vehicle occupant is to change the display position and direction of the frame figure F displayed on the display screen of the display device 2 so as to be superimposed on the overhead view video. It is determined whether or not an operation has been performed. Here, in the process immediately after the display of the bird's-eye view video is started, there is no operation by the vehicle occupant, so a negative determination is made in step S101 and the process proceeds to step S102.

  In this case, in step S102, the frame graphic display position determination unit 14 determines the display position and direction of the frame graphic F to be a predetermined reference position and direction set in advance. In step S <b> 103, the frame graphic drawing unit 15 performs a process of drawing the frame graphic F in a predetermined direction at a predetermined reference position on the overhead image held in the output frame buffer 12. As a result, a bird's-eye view image in which the frame figure F is superimposed in a predetermined direction on a predetermined reference position is displayed on the display screen of the display device 2.

  On the other hand, in the processing cycle after the display of the bird's-eye view video is started and the bird's-eye view image in which the frame figure F is superimposed in a predetermined direction on a predetermined reference position on the display screen of the display device 2 is displayed, If an operation for changing the display position or direction of the frame graphic F on the overhead video is performed using the input device 4, the determination in step S101 is affirmative and the process proceeds to step S104.

  In this case, in step S <b> 104, the frame graphic display position determination unit 14 determines the display position and direction of the frame graphic F according to the contents of the operation input of the vehicle occupant input from the operation input device 4. In step S105, the frame graphic drawing unit 15 performs processing for drawing the frame graphic F in the direction determined in step S104 at the display position determined in step S104 on the overhead view video held in the output frame buffer 12. As a result, a bird's-eye view image in which the frame figure F is superimposed on the display screen of the display device 2 in a direction corresponding to the operation of the vehicle occupant is displayed at a position corresponding to the operation of the vehicle occupant.

  Next, in step S106, it is determined whether or not an instruction to confirm the display position and direction of the frame figure F on the display screen of the display device 2 is input by an operation using the operation input device 4 of the vehicle occupant. Is done. And while the instruction | indication which determines the display position and direction of the frame figure F is not input, the processing flow of FIG. 8 is complete | finished as it is. On the other hand, when an instruction to confirm the display position and direction of the frame graphic F is input, the process proceeds to the next step S107.

  If the determination in step S106 is affirmative and the process proceeds to step S107, in step S107, the target parking space specifying unit 16 displays the frame graphic F at the position where the frame graphic F is currently displayed on the display screen of the display device 2. It recognizes that the target parking space exists in the direction in which F is displayed, and specifies that as the target parking space.

  Next, in step S108, the target trajectory calculation unit 17 calculates an ideal vehicle trajectory from the current position of the vehicle to the target parking space specified in step S107 as the target trajectory. In step S109, the target trajectory line drawing unit 18 performs processing for drawing the target trajectory line L1 on the bird's-eye view image held in the output frame buffer 12 based on the target trajectory information calculated in step S108. Do.

  Further, in step S110, the predicted trajectory calculation unit 19 uses the current steering operation angle and the traveling direction of the vehicle to move the trajectory when the vehicle is moved from the current position along the traveling direction at the same steering operation angle. Calculate as the expected trajectory. Then, in step S111, the predicted trajectory line drawing unit 20 performs a process of drawing the predicted trajectory line L2 on the bird's-eye view video held in the output frame buffer 12 based on the predicted trajectory information calculated in step S110. The processing flow of FIG. 8 is finished.

  Through the processing from step S107 to step S111, the target track line L1 that serves as an index for moving the vehicle from the current position to the target parking space, or the vehicle from the current position is displayed on the display screen of the display device 2. A bird's-eye view image on which the expected trajectory line L2 representing the trajectory when moved as it is is superimposed is displayed. Therefore, the driver of the vehicle operates the steering of the vehicle so that the predicted trajectory line L2 overlaps the target trajectory line L1 while watching the display screen of the display device 2, and performs an operation of gradually reversing the vehicle. The vehicle can be appropriately parked in the target parking space.

  As described above in detail with reference to specific examples, the parking assistance device according to the present embodiment has a bird's-eye view when converting the captured image of the rear camera 1 into a bird's-eye view image and displaying it on the display screen of the display device 2. At the start of video display, a frame figure F having a size corresponding to the outer shape of the vehicle is displayed on a predetermined reference position on the display screen so as to be superimposed on the overhead video in a predetermined direction, and the frame is displayed in accordance with an operation by the vehicle occupant. The display position and direction of the figure F can be changed, and the target parking space that is the target for parking the vehicle is specified based on the display position and direction of the frame figure F after the change. Therefore, according to the parking assistance device of the present embodiment, the vehicle occupant has a sufficient area that allows the vehicle to be parked on the bird's-eye view image on the basis of the frame figure F displayed on the display screen of the display device 2. If there is a parking space that can be parked, the target parking space can be designated by a very simple and intuitive operation such as moving the frame figure F to that position on the display screen. Improved operability when specifying a space.

  Moreover, according to the parking assistance apparatus of this embodiment, when a target parking space is specified on the bird's-eye view image displayed on the display screen of the display device 2, the vehicle is moved from the current position to the target parking space. The target trajectory line L1 that is an index at the time and the predicted trajectory line L2 that represents the trajectory when the vehicle is moved from the current position as it is are superimposed on the overhead image and displayed on the display screen. The driver operates the vehicle steering so that the predicted track line L2 overlaps the target track line L1 while looking at the display screen of the display device 2, thereby gradually moving the vehicle backward, so that the target parking space is obtained. The vehicle can be parked appropriately, and the driving operation of the driver when the vehicle is parked can be appropriately supported.

  In the parking assistance device of the present embodiment described above, the target track line L1 and the predicted track line L2 are displayed on the display screen of the display device 2, and the expected track line L2 on the display screen is displayed to the vehicle driver. Is operated by steering the vehicle so as to overlap the target track line L1, and the driver's driving operation is supported, but the target parking space is displayed on the overhead view image displayed on the display screen of the display device 2. By calculating the target trajectory from the current position of the vehicle to the target parking space and automatically adjusting the steering operation angle of the vehicle so that the vehicle moves along the target trajectory. You may make it support the driver | operator's driving operation at the time of parking. In this case, since the driver of the vehicle can park the vehicle in the target parking space only by appropriately operating the accelerator pedal and the brake pedal, the driving operation by the driver becomes simpler.

[Second Embodiment]
Next, as a second embodiment of the present invention, images of the vehicle front, rear, and left and right side regions are respectively captured by four in-vehicle cameras, and the captured images captured by these four in-vehicle cameras are overhead images. A parking support device that displays a continuous view of the surrounding area of the vehicle as a single overhead view of the surroundings of the vehicle as viewed from a virtual viewpoint above the vehicle. An example in which the present invention is applied will be described. In the following description, the same components as those in the first embodiment are denoted by the same reference numerals in the drawings, and detailed description thereof is omitted.

  FIG. 9 shows a schematic configuration of the parking assistance apparatus of the present embodiment. The parking assist apparatus shown in FIG. 9 includes an image processing unit 30 that creates a vehicle surroundings overhead view image to be displayed as a main component, and a front camera 21, a rear camera 22, Four vehicle-mounted cameras, the right side camera 23 and the left side camera 24, are connected to each other, and the display device 2 is connected to the output side. In addition to the four in-vehicle cameras 21, 22, 23, and 24, the display changeover switch 3, the operation input device 4, and the vehicle information acquisition device 5 are connected to the input side of the image processing unit 10, respectively.

  The four in-vehicle cameras, that is, the front camera 21, the rear camera 22, the right side camera 23, and the left side camera 24 are, for example, wide-angle cameras having an angle of view of about 180 degrees, and these four in-vehicle cameras all surround the vehicle. It is mounted in the right place of the vehicle so that the image of the area can be taken. Specifically, as shown in FIG. 10, the front camera 21 is installed in a front bumper or the like in front of the vehicle, and shoots an image of a predetermined imaging area A1 in front of the vehicle in a direction looking down obliquely with respect to the ground. The rear camera 22 is installed in a rear bumper or the like at the rear of the vehicle, and shoots an image of a predetermined shooting area A2 at the rear of the vehicle in a direction looking down obliquely with respect to the ground. Further, the right side camera 23 is installed on a side mirror on the right side of the vehicle, and shoots an image of a predetermined shooting area A3 on the right side of the vehicle in a direction looking down obliquely with respect to the ground. Furthermore, the left side camera 24 is installed in a side mirror on the left side of the vehicle, and shoots an image of a predetermined shooting area A4 on the left side of the vehicle in a direction looking down obliquely with respect to the ground. These four in-vehicle cameras 21, 22, 23, and 24 take images of predetermined areas A1, A2, A3, and A4 in accordance with instructions from the image processing unit 30, respectively. The signal is converted into a signal and output to the image processing unit 30 through a predetermined analog communication line as, for example, a composite signal (an analog format video signal).

  The image processing unit 30 represents a state in which the entire area around the vehicle is looked down from a virtual viewpoint above the vehicle, using captured images of the four on-vehicle cameras of the front camera 21, the rear camera 22, the right side camera 23, and the left side camera 24. Create a bird's-eye view of the surroundings of the vehicle and use the first frame to show the target parking space in the vehicle's overhead view, and the second frame to show the recommended parking start position where you can park without turning back to the target parking space. A second target trajectory line representing the target trajectory from the parking start position to the target parking space, a second expected trajectory line representing the expected trajectory from the recommended parking start position, and the like are superimposed as necessary to provide an overhead view image to be displayed. This is output to the display device 2, for example, a video processing circuit composed of various electric / electronic elements, or a digital circuit composed of a semiconductor chip. It is constituted by such as Le image processing circuit.

  In particular, as shown in FIG. 11, the image processing unit 30 temporarily captures captured images input as video signals from four in-vehicle cameras 21, 22, 23, and 24 for each frame. Four input frame buffers 31a, 31b, 31c, and 31d to be held, an output frame buffer 32 to temporarily hold the video data of the vehicle surroundings overhead view image to be output to the display device 2, and an input The necessary video areas are cut out from the captured images of the in-vehicle cameras 21, 22, 23, and 24 held in the frame buffers 31a, 31b, 31c, and 31d, converted into overhead images, and output as one continuous vehicle surroundings overhead image. And a viewpoint conversion processing unit 33 for writing to the frame buffer 32. Further, the image processing unit 30 includes a first frame graphic display position determination unit 34 that determines the display position and direction of the first frame graphic to be displayed on the display device 2 while being superimposed on the vehicle surroundings overhead view video, and an output frame buffer. The first frame graphic drawing unit 35 for drawing the first frame graphic on the vehicle surroundings overhead view video held in 32, the target parking space specifying unit 36 for specifying the target parking space for parking the vehicle, and the target parking space A recommended parking start position calculation unit 37 that calculates a recommended parking start position that can be parked without performing a turn-back operation, and a second parking frame that indicates the recommended parking position is drawn on the vehicle surroundings overhead image held in the output frame buffer 32. The second frame graphic drawing unit 38, the second target trajectory calculation unit 39 for calculating the target trajectory from the recommended parking start position to the target parking space, and the output frame buffer 12 The second target trajectory line drawing unit 40 for drawing the second target trajectory line on the vehicle surroundings bird's-eye view video, the second predicted trajectory calculation unit 41 for calculating the predicted trajectory from the recommended parking start position, and the output frame buffer 12 A second predicted trajectory line drawing unit 42 is provided for drawing the second predicted trajectory line on the held vehicle surroundings overhead view video.

  The video signals of the captured images output from the four on-vehicle cameras of the front camera 21, rear camera 22, right side camera 23, and left side camera 24 are each A / D converted and decoded into RGB signals. The frame data constituting the frames of the captured images of the in-vehicle cameras 21, 22, 23, and 24 are stored in the input frame buffers 31a, 31b, 31c, and 31d of the image processing unit 30, respectively. Then, the viewpoint conversion processing unit 33 of the image processing unit 30 converts, for example, a conversion map that describes the correspondence between the addresses of the input frame buffers 31a, 31b, 31c, and 31d for each pixel and the addresses of the output frame buffer 32 for each pixel. Is used to extract a necessary video area from the captured images of the in-vehicle cameras 21, 22, 23, 24 stored in the input frame buffers 31a, 31b, 31c, 31d and look down from a virtual viewpoint above the vehicle. The video data is processed and connected, and the video data is stored in the output frame buffer 32 as a series of vehicle surrounding bird's-eye views showing the entire vehicle surroundings.

  The video data (RGB signal) of the vehicle surroundings overhead view image stored in the output frame buffer 32 is encoded into a composite signal and D / A converted at a timing synchronized with the shooting period of each of the in-vehicle cameras 21, 22, 23, and 24. Then, it is output from the image processing unit 30 and output to the display device 2 via a predetermined analog communication line. Then, the display device 2 performs a display operation based on the video signal from the image processing unit 30, so that the entire area around the vehicle is displayed on the display screen of the display device 2, for example, as shown in FIG. A vehicle surroundings bird's-eye view image showing a state of looking down from the vehicle is displayed. Note that in the vehicle surroundings overhead view image illustrated in FIG. 12, the video region B1 is a video obtained by cutting out a part of the video captured by the front camera 21 and converting the viewpoint, and the video region B2 is a part of the video captured by the rear camera 22. The video region B3 is a video obtained by cutting out a part of the video shot by the right side camera 23 and converting the viewpoint, and the video region B4 cuts out a part of the video shot by the left side camera 24. This is a video whose viewpoint has been changed. Further, in the vehicle surroundings bird's-eye view image illustrated in FIG. 12, a mask line M is drawn at the boundary of each video region so that the vehicle occupant can recognize that the portion is a joint of the video. In addition, in the vehicle surroundings bird's-eye view image illustrated in FIG. 12, an image C displayed at the center of the image is a vehicle image representing the position of the vehicle, and is created by computer graphics or the like and superimposed on the vehicle surroundings bird's-eye view image. ing.

  Further, in the parking assist device of the present embodiment, the first frame graphic drawing unit 35 of the image processing unit 30 draws the first frame graphic on the vehicle surroundings overhead image held in the output frame buffer 32, thereby displaying the frame. On the display screen of the apparatus 2, a first frame figure having a size corresponding to the outer shape of the vehicle is displayed by being superimposed on the vehicle surroundings overhead view video. When a target parking space that is a target for parking the vehicle is specified, the second frame graphic drawing unit 38 displays one or more second frame graphics on the vehicle surroundings overhead image held in the output frame buffer 32. Is drawn on the display screen of the display device 2 so as to be superimposed on the vehicle surroundings bird's-eye view video and to display a second frame graphic indicating one or more recommended parking start positions. Furthermore, in the parking assistance device of this embodiment, when the vehicle moves to the recommended parking start position indicated by the second frame graphic, the vehicle surroundings overhead view image in which the second target trajectory line drawing unit 40 is held in the output frame buffer 32 is displayed. The second target trajectory line is drawn on top, and the second predicted trajectory line drawing unit 42 draws the second predicted trajectory line on the vehicle surroundings overhead image held in the output frame buffer 32, thereby displaying on the display device 2 On the screen, the second target trajectory line and the second predicted trajectory line are displayed so as to be superimposed on the vehicle surroundings overhead view video.

  The display position and direction of the first frame graphic on the display screen of the display device 2 are determined by the first frame graphic display position determination unit 34 of the image processing unit 30. The first frame graphic display position determination unit 34 determines, for example, that the vehicle has entered a parking posture when the display changeover switch 3 is operated by a vehicle occupant or based on vehicle information acquired by the vehicle information acquisition device 5. When a certain trigger for instructing display start of the vehicle surroundings overhead view image is detected, for example, first, the display position and direction of the first frame figure on the display screen of the display device 2 are determined in advance. Decide on a predetermined reference position and direction. Then, the first frame graphic drawing unit 35 displays the vehicle surroundings overhead image held in the output frame buffer 32 based on the display position and direction information determined by the first frame graphic display position determining unit 34. By drawing the first frame figure, for example, as shown in FIG. 13A, the first corresponding to the outer shape of the vehicle in a predetermined direction at a predetermined reference position on the display screen of the display device 2. The frame figure F1 is displayed superimposed on the vehicle surroundings overhead view video.

  Moreover, in the parking assistance device of the present embodiment, the display position and direction of the first frame graphic F1 on the display screen of the display device 2 can be set at any position by the vehicle occupant performing an operation using the operation input device 4. And it can be changed in the direction. That is, when the vehicle occupant performs an operation of changing the display position and direction of the first frame graphic F1 on the display screen of the display device 2 using the operation input device 4 after the display of the vehicle surroundings overhead view image is started. Information indicating the operation content is input to the first frame graphic display position determination unit 34 of the image processing unit 10. When information indicating the operation content of the vehicle occupant is input from the operation input device 4 to the first frame graphic display position determination unit 34, the first frame graphic display position F1 on the display screen of the display device 2 according to the operation content. Determine the display position and direction. Then, the first frame graphic drawing unit 35 displays the vehicle surroundings overhead image held in the output frame buffer 32 based on the display position and direction information determined by the first frame graphic display position determining unit 34. By drawing the first frame graphic F1, the display position and direction of the first frame graphic F1 on the display screen of the display device 2 are changed. For example, as shown in FIG. The first frame figure F1 having a size corresponding to the outer shape of the vehicle is displayed in an arbitrary direction at an arbitrary display position so as to be superimposed on the vehicle surroundings overhead view video.

  Further, in the parking assistance device of the present embodiment, when the vehicle occupant inputs an instruction to confirm the display position and direction of the first frame graphic F1 on the display screen of the display device 2 using the operation input device 4. The target parking space specifying unit 36 of the image processing unit 30 has the target frame in the direction in which the first frame graphic F1 is displayed at the position where the first frame graphic F1 is displayed on the display screen of the display device 2. Recognizing that there is a parking space to be identified, it is specified as the target parking space.

  Then, when the target parking space is specified by the target parking space specifying unit 36 of the image processing unit 30, the recommended parking start position calculating unit 37 is based on the information indicating the minimum turning radius of the vehicle stored in advance. A position on the vehicle surroundings overhead image that can be parked without performing a return operation to the parking space is calculated as a recommended parking start position. At this time, the recommended parking start position calculation unit 37 may calculate at least one recommended parking start position on the vehicle surroundings overhead image, but for example, an obstacle that hinders the track of the vehicle near the target parking space. In consideration of the fact that there are cases where there is a case, it is desirable to calculate a plurality of recommended parking start positions on the vehicle surroundings overhead view video. Then, the second frame graphic drawing unit 38 has one on the vehicle surroundings overhead view image held in the output frame buffer 32 based on the information of the recommended parking start position calculated by the recommended parking start position calculating unit 37. Alternatively, by drawing a plurality of second frame figures, for example, as shown in FIG. 14, the second frame figure F2 indicating the recommended parking start position is superimposed on the vehicle surroundings overhead view image on the display screen of the display device 2. Will be displayed. In FIG. 14, the recommended parking start position calculation unit 37 calculates three recommended parking start positions, and the second frame graphic drawing unit 38 draws three second frame graphics F2 indicating these recommended parking start positions. An example is shown.

  Moreover, in the parking assistance device of the present embodiment, the recommended parking start position indicated as the second frame graphic F2 on the vehicle surroundings bird's-eye view image by the driving operation of the vehicle driver with reference to the display screen of the display device 2. When the vehicle moves, the second target trajectory calculation unit 39 sets, as the target trajectory, an ideal trajectory that can move from the recommended parking start position, which is the current position of the vehicle, to the target parking space, for example, with a constant steering operation angle. calculate. Then, the second target trajectory line drawing unit 40 displays the target trajectory on the vehicle surroundings overhead image held in the output frame buffer 32 based on the target trajectory information calculated by the second target trajectory calculation unit 39. For example, as illustrated in FIG. 15A, the second target serving as an index when the vehicle is moved from the recommended parking start position to the target parking space on the display screen of the display device 2 by drawing the line to be represented. The track line L11 is displayed superimposed on the vehicle surroundings overhead view video. In addition, when the vehicle is moved to the recommended parking start position, if the vehicle surroundings bird's-eye view image is displayed on the display screen of the display device 2 at the same scale, the target parking space may be off the display screen. It is desirable that the scale of the vehicle surroundings overhead image displayed on the display screen of the display device 2 is changed so that the target parking space is within the display screen.

  In addition, when the vehicle moves to the recommended parking start position indicated as the second frame figure F2 on the vehicle surroundings overhead view image, the second predicted trajectory calculation unit 41 determines the steering operation angle acquired by the vehicle information acquisition device 5. Based on the information and the shift position information, the current steering operation angle and the traveling direction of the vehicle are determined, and based on that, the vehicle is moved from the recommended parking start position at the same steering operation angle along the traveling direction. The trajectory is calculated as an expected trajectory. Then, the second predicted trajectory line drawing unit 42 displays the predicted trajectory on the vehicle surroundings overhead image held in the output frame buffer 32 based on the predicted trajectory information calculated by the second predicted trajectory calculation unit 41. For example, as illustrated in FIG. 15A, the second predicted trajectory line L <b> 12 that represents the trajectory when the vehicle is moved from the recommended parking start position as it is on the display screen of the display device 2, for example, is drawn. However, it is displayed superimposed on the vehicle surroundings overhead view video.

  In the parking assistance device of the present embodiment, as described above, the second frame indicating the recommended parking start position when the target parking space is specified on the vehicle surroundings overhead image displayed on the display screen of the display device 2. When the figure is displayed on the display screen and the vehicle moves to the recommended parking start position, the vehicle to the target parking space is displayed by displaying the second target track line L11 and the second predicted track line L12 on the display screen. In order to facilitate the parking of the vehicle, the driving operation by the vehicle driver is supported. That is, the driver of the vehicle moves the vehicle to the recommended parking start position while viewing the display screen of the display device 2 as shown in FIG. 14, for example, and then displays the display screen of the display device 2 as shown in FIG. 15B, the steering of the vehicle is operated so that the second predicted trajectory line L12 reflecting the current steering operation angle of the vehicle overlaps the second target trajectory line L11, and the display screen of FIG. If the operation of gradually moving the vehicle backward is performed in a state where the vehicle surroundings overhead view image as shown in FIG. 3 is displayed, the vehicle can be parked appropriately in the target parking space.

  Here, in the parking assistance device of the present embodiment, the flow of a series of processing by the image processing unit 30 when displaying the vehicle surroundings overhead view video on the display screen of the display device 2 will be described with reference to the flowcharts of FIGS. 16 and 17. While explaining. The processing flow of FIG. 16 is a main routine repeatedly executed by the image processing unit 30 in synchronization with the shooting periods of the four in-vehicle cameras 21, 22, 23, and 24. The processing flow of FIG. It is a subroutine which shows the detail of the drawing process in step S23.

  The processing flow of FIG. 16 is, for example, when there is an operation of the display changeover switch 3 by a vehicle occupant, or when it is determined that the vehicle has entered a parking posture based on the vehicle information acquired by the vehicle information acquisition device 5. This is started when a trigger for instructing display start of the vehicle surroundings overhead view video is detected.

  When the processing flow of FIG. 16 is started, first, in step S21, captured images respectively captured by the four on-vehicle cameras of the front camera 21, the rear camera 22, the right side camera 23, and the left side camera 24 are image processing apparatuses. 30 and stored in the input frame buffers 31a, 31b, 31c and 31d as video data for each frame. Next, in step S22, the viewpoint conversion processing unit 33 is necessary from the captured images of the in-vehicle cameras 21, 22, 23, 24 stored in the input frame buffers 31a, 31b, 31c, 31d using a conversion map or the like. A video area is extracted and converted into a bird's-eye view image looking down from a virtual viewpoint above the vehicle, and these are connected to form a series of vehicle surrounding bird's-eye view images showing the entire vehicle periphery, and the image data is output to the frame buffer. 32.

  Next, in step S23, the first frame graphic drawing unit 35 performs a process of drawing the first frame graphic F1 on the vehicle surroundings overhead view video held in the output frame buffer 32, and the second frame graphic if necessary. The graphic drawing unit 38 performs a process of drawing the second frame pattern F2, the second target trajectory line drawing unit 40 drawing the second target trajectory line L11, and the second predicted trajectory line drawing unit 42 drawing the second predicted trajectory line L12. In step S <b> 24, the video data of the vehicle surroundings overhead view video to be displayed held in the output frame buffer 32 is output from the image processing device 30 to the display device 2. As a result, a vehicle surroundings overhead view image in which the first frame graphic F1, the second frame graphic F2, the second target trajectory line L11, the second predicted trajectory line L12, and the like are superimposed is displayed on the display screen of the display device 2. It will be.

  Note that the processing flow of FIG. 16 includes four processes until the vehicle is parked and a trigger for instructing to stop displaying the vehicle surroundings overhead image is detected, or until the parking assist device is turned off. The process is repeated in synchronization with the imaging cycle of the in-vehicle cameras 21, 22, 23, and 24, and the process ends when an instruction to stop the display of the vehicle surroundings bird's-eye view video or the power-off of the parking support apparatus is performed.

  The drawing process of the first frame graphic F1, the second frame graphic F2, the second target trajectory line L11, and the second predicted trajectory line L12 in step S23 of FIG. 16 is performed according to the processing flow of FIG. When the processing flow in FIG. 17 is started, first, in step S201, the display position and direction of the first frame graphic F1 displayed on the display screen of the display device 2 in a superimposed manner on the vehicle surroundings overhead view image is changed. It is determined whether or not an operation by the vehicle occupant has been performed. Here, in the process immediately after the display of the vehicle surroundings overhead view image is started, there is no operation by the vehicle occupant, so a negative determination is made in step S201, and the process proceeds to step S202.

  In this case, in step S202, the first frame graphic display position determination unit 34 determines the display position and direction of the first frame graphic F1 to a predetermined reference position and direction set in advance. In step S <b> 203, the first frame graphic drawing unit 35 performs a process of drawing the first frame graphic F <b> 1 in a predetermined direction at a predetermined reference position on the vehicle surroundings overhead image held in the output frame buffer 32. Thereby, on the display screen of the display device 2, a vehicle surroundings overhead image in which the first frame figure F1 is superimposed at a predetermined reference position in a predetermined direction is displayed.

  On the other hand, in the processing cycle after displaying the vehicle surroundings bird's-eye view image and displaying the vehicle surroundings bird's-eye view image in which the first frame graphic F1 is superimposed in a predetermined direction on a predetermined reference position on the display screen of the display device 2. If the vehicle occupant performs an operation to change the display position or direction of the first frame graphic F1 on the vehicle surroundings overhead view image using the operation input device 4, the determination in step S201 is affirmative and the process proceeds to step S204.

  In this case, in step S204, the first frame graphic display position determination unit 34 determines the display position and direction of the first frame graphic F1 according to the contents of the operation input of the vehicle occupant input from the operation input device 4. . In step S205, the first frame graphic drawing unit 35 places the first frame graphic F1 in the direction determined in step S204 at the display position determined in step S204 on the vehicle surroundings overhead view video held in the output frame buffer 32. Perform the drawing process. Thereby, on the display screen of the display device 2, a vehicle surroundings overhead image in which the first frame figure F1 is superimposed at a position corresponding to the operation of the vehicle occupant in a direction corresponding to the operation of the vehicle occupant is displayed. Become.

  Next, in step S206, whether or not an instruction to confirm the display position and direction of the first frame graphic F1 on the display screen of the display device 2 is input by an operation using the operation input device 4 of the vehicle occupant. Is judged. Then, while the instruction to confirm the display position and direction of the first frame graphic F1 is not input, the processing flow of FIG. On the other hand, when an instruction to confirm the display position and direction of the first frame graphic F1 is input, the process proceeds to the next step S207.

  If the determination in step S206 is affirmative and the process proceeds to step S207, in step S207, the target parking space specifying unit 36 is located at the position where the first frame graphic F1 is currently displayed on the display screen of the display device 2. It recognizes that the target parking space exists in the direction in which the first frame graphic F1 is displayed, and specifies it as the target parking space.

  Next, in step S208, the recommended parking start position calculation unit 37 is based on the information indicating the minimum turning radius of the vehicle and can be parked without performing a turning operation to the target parking space specified in step S207. The position on the image is calculated as the recommended parking start position. In step S209, the second frame figure drawing unit 38 starts recommended parking on the vehicle surroundings overhead image held in the output frame buffer 32 based on the information of the recommended parking start position calculated in step S208. A process of drawing the second frame figure F2 indicating the position is performed. Thereby, on the display screen of the display device 2, a vehicle surroundings bird's-eye view image in which the second frame figure F <b> 2 is superimposed at the recommended parking start position where parking is possible without performing a turning operation is displayed.

  Next, in step S210, for example, calculation is performed in step S208 based on various vehicle information acquired by the vehicle information acquisition device 5, current position information of the vehicle detected by a position detection function such as an in-vehicle navigation device, and the like. It is determined whether or not the vehicle has moved to the recommended parking start position. Then, while the vehicle has not moved to the recommended parking start position, the processing flow of FIG. On the other hand, when the vehicle has moved to the recommended parking start position, the process proceeds to the next step S211.

  If the determination in step S210 is affirmative and the process proceeds to step S211, in step S211, the second target trajectory calculation unit 39 is ideal from the recommended parking start position where the vehicle has moved to the target parking space specified in step S207. The vehicle trajectory is calculated as the target trajectory. In step S212, the second target trajectory line drawing unit 40 displays the second target trajectory line on the vehicle surroundings overhead image held in the output frame buffer 32 based on the target trajectory information calculated in step S212. Processing to draw L11 is performed.

  In step S213, the second predicted trajectory calculation unit 41 moves the vehicle along the traveling direction from the recommended parking start position at the steering operation angle as it is based on the current steering operation angle and the traveling direction of the vehicle. The time trajectory is calculated as the expected trajectory. In step S214, the second predicted trajectory line drawing unit 42 displays the second predicted trajectory line on the vehicle surroundings overhead image held in the output frame buffer 32 based on the predicted trajectory information calculated in step S213. The process of drawing L12 is performed, and the process flow of FIG.

  Through the processing from step S211 to step S214, the second target track line L11 serving as an index for moving the vehicle from the recommended parking start position to the target parking space on the display screen of the display device 2, the vehicle The vehicle surroundings bird's-eye view image on which the second predicted trajectory line L12 representing the trajectory when the vehicle is moved as it is from the recommended parking start position is displayed. Therefore, the driver of the vehicle operates the vehicle to gradually move the vehicle by operating the steering of the vehicle so that the second predicted track line L12 overlaps the second target track line L11 while looking at the display screen of the display device 2. By doing so, the vehicle can be appropriately parked in the target parking space.

  As described above in detail with reference to specific examples, the parking assist device according to the present embodiment converts the captured images of the four in-vehicle cameras 21, 22, 23, and 24 into a bird's-eye view and connects them to the vehicle. When displaying on the display screen of the display device 2 as a surrounding bird's-eye view video, at the start of displaying the vehicle surrounding bird's-eye view image, a first frame figure F1 having a size corresponding to the outer shape of the vehicle is set at a predetermined reference position on the display screen. The display position of the first frame graphic F1 after the change is displayed so that the display position and direction of the first frame graphic F1 can be changed according to the operation by the vehicle occupant. And the target parking space used as the target which parks a vehicle is specified based on and direction. Therefore, according to the parking assistance device of the present embodiment, the vehicle occupant is sufficiently capable of parking the vehicle on the vehicle surroundings bird's-eye view image on the basis of the first frame graphic F1 displayed on the display screen of the display device 2. If there is a parking space that can be parked and there is a parking space that can be parked, the target parking space is designated by an extremely simple and intuitive operation such as moving the first frame figure F1 to the position on the display screen. This improves the operability when designating the target parking space.

  Moreover, according to the parking assistance apparatus of this embodiment, when a target parking space is specified on the vehicle surroundings bird's-eye view image currently displayed on the display screen of the display apparatus 2, it will go to a target parking space, without performing a turning operation. Since the recommended parking start position where parking is possible is superimposed on the vehicle surroundings overhead image as the second frame figure F2 and displayed on the display screen, the driver of the vehicle looks at the display screen of the display device 2. However, the vehicle can be moved to an appropriate parking start position, and the driving operation of the driver when the vehicle is parked can be appropriately supported.

  Furthermore, according to the parking assist device of the present embodiment, when the vehicle moves to the recommended parking start position, the second target trajectory line L11 serving as an index for moving the vehicle from the recommended parking start position to the target parking space, Since the second predicted trajectory line L12 representing the trajectory when the vehicle is moved as it is from the recommended parking start position is superimposed on the vehicle surroundings overhead image and displayed on the display screen, the driver of the vehicle While looking at the display screen of the display device 2, the steering of the vehicle is operated so that the second predicted trajectory line L12 overlaps the second target trajectory line L11, and the vehicle is gradually moved, so that the target parking space is obtained. The vehicle can be parked appropriately, and the driving operation of the driver when the vehicle is parked can be appropriately supported.

  Furthermore, in the parking assistance device of the present embodiment, one continuous vehicle surroundings bird's-eye view image showing a state of looking down from the virtual viewpoint above the vehicle is displayed on the display screen of the display device 2, Since the target parking space can be specified by performing an operation of moving the first frame figure F1 to a desired position on the vehicle surroundings overhead view image, for example, as shown in FIG. 18, it is desired to specify the target parking space. An operation for designating a parking space can be performed in a state where the vehicle is placed sideways. Thus, for example, in a scene where a vehicle is parked in a parking area where parking spaces that can be parked in parallel on both the left and right sides are sandwiched across the road, such as a service area on an expressway, the vehicle is in a position where the road is not blocked. When the vehicle is stopped, an operation for designating the target parking space can be performed without hindering the traveling of the following vehicle. In addition, it is possible to prevent a situation in which another vehicle parks a place to be designated as the target parking space first while performing the operation of setting the target parking space.

  In the parking assistance device of the present embodiment described above, the second target trajectory line L11 and the second predicted trajectory line L12 are displayed on the display screen of the display device 2, and the vehicle driver is notified on the display screen. The driver's driving operation is assisted by operating the steering of the vehicle so that the second predicted track line L12 overlaps the second target track line L11. However, the second predicted track line L12 is displayed on the display screen of the display device 2. When the target parking space is identified on the vehicle overhead view image, the target trajectory from the current position of the vehicle to the target parking space is calculated, or the target parking space is identified and the vehicle has moved to the recommended parking start position Sometimes, the target trajectory from the recommended parking start position to the target parking space is calculated, and the vehicle steering operation angle is automatically adjusted so that the vehicle moves along these target trajectories. And by, it may be to support the driving operation of the driver when the vehicle parking. In this case, since the driver of the vehicle can park the vehicle in the target parking space only by appropriately operating the accelerator pedal and the brake pedal, the driving operation by the driver becomes simpler.

  The parking support apparatus according to the embodiment of the present invention has been specifically described above. However, the parking support system described above exemplifies an application example of the present invention, and the technical scope of the present invention is as described above. It is needless to say that the present invention is not limited to the contents disclosed in the description of the embodiments, and includes various alternative techniques that can be easily derived from these disclosures.

It is a block diagram which shows schematic structure of the parking assistance apparatus of 1st Embodiment. It is a block diagram which shows the detailed internal structure of an image processing unit. It is a figure explaining an overhead image, (a) is a figure which shows an example of the overhead image displayed on a display apparatus, (b) is a figure which shows the picked-up image of the rear camera before converting into an overhead image. It is a figure which shows a mode that the frame figure was superimposed on a bird's-eye view image, (a) is a figure which shows a mode that the frame figure was displayed in a predetermined direction at a predetermined reference position, and (b) is a vehicle occupant. It is a figure which shows a mode that the display position and direction of the frame figure were changed according to operation. It is a schematic diagram explaining the operation procedure at the time of changing the display position and direction of a frame figure on a display screen. It is a figure which shows a mode that the target track line and the predicted track line were displayed superimposed on the bird's-eye view image, (a) is a diagram showing a state where the target track line and the predicted track line do not overlap, and (b) is a vehicle It is a figure which shows a mode that the expected track line overlapped with the target track line according to a passenger | crew's steering operation. In the parking assistance apparatus of 1st Embodiment, it is a flowchart which shows the flow of a series of processes by the image processing unit at the time of displaying a bird's-eye view image | video on the display screen of a display apparatus. It is a flowchart which shows the flow of the drawing process in step S13 of FIG. It is a block diagram which shows schematic structure of the parking assistance apparatus of 2nd Embodiment. It is a figure explaining the to-be-photographed area | region image | photographed with the installation position of each vehicle-mounted camera and each vehicle-mounted camera in the parking assistance apparatus of 2nd Embodiment. It is a block diagram which shows the detailed internal structure of an image processing unit. It is a figure which shows one display example of the vehicle surroundings bird's-eye view image | video displayed on a display apparatus. (A) is a figure which shows a mode that the 1st frame figure was displayed on the vehicle surroundings bird's-eye view image, (a) is a figure which shows a mode that the 1st frame figure was displayed on the predetermined reference position in the predetermined direction, (b) is a figure which shows a mode that the display position and direction of the 1st frame figure were changed according to operation of the vehicle passenger. It is a figure which shows a mode that the several 2nd frame figure which shows a some recommended parking start position on a vehicle surroundings bird's-eye view image | video was displayed. It is a figure which shows a mode that the 2nd target track line and the 2nd predicted track line were displayed superimposed on the vehicle surroundings bird's-eye view image, and (a) does not overlap the 2nd target track line and the 2nd predicted track line. The figure which shows a mode, (b) is a figure which shows a mode that the 2nd expected track line overlapped with the 2nd target track line according to steering operation of the vehicle passenger. In the parking assistance apparatus of 2nd Embodiment, it is a flowchart which shows the flow of a series of processes by the image processing unit at the time of displaying a bird's-eye view image | video on the display screen of a display apparatus. It is a flowchart which shows the flow of the drawing process in step S23 of FIG. It is a figure explaining the effect of the parking assistance apparatus of 2nd Embodiment, and is a figure which shows a mode that operation which designates a parking space in the state which laid the vehicle on the place which wants to designate as a target parking space is performed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Rear camera 2 Display apparatus 3 Display changeover switch 4 Operation input device 5 Vehicle information acquisition apparatus 10 Image processing unit 11 Input frame buffer 12 Output frame buffer 13 View point conversion process part 14 Frame figure display position determination part 15 Frame figure drawing part 16 Target Parking space identification unit 17 Target trajectory calculation unit 18 Target trajectory line drawing unit 19 Expected trajectory line calculation unit 20 Expected trajectory line drawing unit 21 Front camera 22 Rear camera 23 Right side camera 24 Left side camera 30 Image processing units 31a, 31b, 31c, 31d Input frame buffer 32 Output frame buffer 33 View point conversion processing unit 34 First frame graphic display position determining unit 35 First frame graphic drawing unit 36 Target parking space specifying unit 37 Recommended parking start position calculating unit 38 Second frame graphic drawing unit 39 Second target trajectory calculation 40 second target trajectory line drawing unit 41 the second predicted trajectory calculating section 42 second predicted trajectory line drawing unit

Claims (5)

The video around the vehicle captured by multiple in-vehicle cameras is converted into a video looking down from a virtual viewpoint above the vehicle and connected to display it as one continuous vehicle surroundings overhead video on the display screen of the display device. In the parking assistance device that assists the driving operation when the vehicle is parked in the target parking space,
Based on the minimum turning radius of the vehicle, a recommended parking start position calculating means for calculating a recommended parking start position that can be parked without switching back to the target parking space;
A recommended parking start position superimposing display means for displaying a figure representing the recommended parking start position superimposed on the vehicle surroundings overhead view image;
The recommended parking start position calculating means calculates a plurality of the recommended parking start positions,
The recommended parking start position superimposed display means, said plurality of recommended parking start position parked vehicle supporting device you characterized thereby displayed superimposed plurality of shapes to the vehicle around overhead image representing the. Target trajectory calculating means for calculating a target trajectory from the previous recommended parking start position to the target parking space;
According to claim 1, characterized in that it comprises a target trajectory line superimposed display means for displaying a line representing the target trajectory calculated by the target track calculating section to overlap the vehicle surroundings bird's-eye-view video as a target trajectory line Parking assistance device. An expected trajectory calculating means for calculating an expected trajectory from the recommended parking start position based on a current steering operation angle and a traveling direction of the vehicle;
3. The predicted trajectory line superimposing display means for displaying the line representing the predicted trajectory calculated by the predicted trajectory calculating means as an expected trajectory line superimposed on the vehicle surroundings overhead view image. The parking assistance device described. Target trajectory calculation means for calculating a target trajectory from the recommended parking start position to the target parking space;
Wherein as the vehicle along the target trajectory is moved, the parking assist apparatus according to claim 1, characterized in that it comprises a steering automatic adjustment means for automatically adjusting the steering angle of the vehicle. The video around the vehicle captured by multiple in-vehicle cameras is converted into a video looking down from a virtual viewpoint above the vehicle and connected to display it as one continuous vehicle surroundings overhead video on the display screen of the display device. Is a parking support method for supporting driving operation when the vehicle is parked in the target parking space,
A first step of calculating a plurality of recommended parking start positions that can be parked without performing a return operation to the target parking space based on a minimum turning radius of the vehicle;
Parking assist method comprising Rukoto and a second step of displaying by superimposing a plurality of figures representing the plurality of recommended parking start position in the vehicle surroundings bird's-eye-view video.

Images