JP2012076483A - Parking support device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To easily and accurately set a parking target for supporting parking in response to operation by a user.SOLUTION: First operation of the user to a set icon P displayed as a set button for setting the parking target is detected, operation succeeding to the first operation, that is, second operation of the user to a desired parking position E on a display image displayed on a monitor device is detected, setting indication of the parking target by the user is accepted, a space capable of parking a vehicle 90 in a corresponding area to the desired parking position E on the display image in an actual space is set as the parking target, and the parking target is set by using a detection result of a peripheral detection device for detecting a peripheral state of the vehicle.

Description

  The present invention relates to a parking assistance device that assists a driving operation when a vehicle is parked.

  Various parking assist technologies have been proposed and put into practical use in order to reduce the driver's driving operation load when parking a vehicle. In order to perform appropriate parking assistance, it is desirable to set a parking target with high accuracy. The parking target is often displayed on the display by a rectangular frame (parking target frame) that is approximately the same size as the vehicle, taking into account the ease of understanding of the driver and other users. In JP 2007-118904 A (Patent Document 1), a parking target frame is moved to a coordinate point designated by a touch operation on a display provided with a touch panel. At this time, the rotation center of the parking target frame is set at this coordinate point, and the user can rotate the parking target frame by operating the touch panel. Thereby, the user can set a parking target frame easily with comparatively high precision (patent document 1: 25th-31st paragraph, FIGS. 3-5 etc.).

  However, the rotation center corresponds to the corner of the actual parking frame as shown in the 30th paragraph of Patent Document 1 and FIGS. Since the display of the monitor device mounted on the vehicle is generally about 7 to 13 inches, the driver is required to perform a relatively delicate operation on the touch panel.

JP 2007-118904 A

  In view of the above-described background, a technique for easily and accurately setting a parking target for parking assistance with an operation by a user is desired.

In view of the above problems, the characteristic configuration of the parking assistance device according to the present invention is as follows.
A display image providing unit that provides a monitor image with a display image including a captured image captured by the in-vehicle camera and a setting button superimposed on the captured image;
On the display image displayed on the monitor device, the first operation of the user with respect to the setting icon displayed as the setting button for detecting a parking target is detected and the operation is subsequent to the first operation. A target setting instruction receiving unit that detects a second operation of the user with respect to a desired parking position and receives a setting instruction of the parking target by the user;
A space in which the vehicle can be parked in a region in an actual space corresponding to the desired parking position on the display image is used as the parking target using a detection result of a surrounding detection device that detects a situation around the vehicle. A parking target setting unit to be set.

  According to this configuration, since the setting icon is selected or designated by the first operation of the user, the parking assistance device can detect that the user is about to set the parking target. A position where the user wants to park the vehicle can be detected by the user's second operation following the first operation. A parking target setting part sets a parking target from the detection result of a periphery detection apparatus based on the position given from the user by 2nd operation. Since the detailed position of the parking target is identified by the parking target setting unit, the user only needs to perform the second operation at the approximate position. Therefore, according to this configuration, it is possible to easily and accurately set a parking target for parking assistance with an operation by the user.

  Here, the first operation of the parking assistance device according to the present invention is performed by a designation operation that designates the setting icon as an operation target, and continues the designation operation and moves the setting icon on the display image. It is a drag operation, and the second operation is preferably a drop operation for releasing the setting icon moved by the drag operation at the desired parking position on the display image. The user can move a setting icon for setting a parking target to a desired position on the screen of the monitor device using an operation unit such as a pointing device or a finger. From the first operation to the second operation, the setting icon or the pointer indicating the setting icon moves continuously on the screen without interruption, so that the parking target can be easily set by a series of operations.

  Alternatively, the first operation of the parking assistance device according to the present invention is a designation operation for designating the setting icon as an operation target, and the second operation is performed within a predetermined time from the first operation. It may be a designation operation for designating the desired parking position on the display image. The monitoring device in the vehicle is not arranged in front of a user such as a driver. For this reason, it may be difficult to move the setting icon continuously on the screen without interruption from the first operation to the second operation. If a setting icon is selected or designated by the first operation and a desired parking position is set or designated by the second operation within a predetermined time from the first operation, a parking target is set without a continuous operation. be able to.

  Here, if the monitor device is a monitor device with a touch panel, the setting button displayed as the setting icon can be displayed as a touch button. In this case, the operation for designating the setting icon as an operation target and the designating operation for designating the desired parking position can be a touch operation by the user. Specifically, the first operation is a drag operation in which the setting icon is touched, the touch is continued and the setting icon is moved on the display image, and the second operation is performed in the display It can be a drop operation for releasing the setting icon being moved by the drag operation at the desired parking position on the image. Further, the first operation is a touch operation on the setting icon, and the second operation is a touch operation on the desired parking position on the display image within a predetermined time from the first operation. be able to. As described above, if a touch operation on the touch button is used, a parking target can be easily set without using a special pointing device.

  In addition, the parking target setting unit of the parking assistance device according to the present invention recognizes the parking target by recognizing a road marking indicating a parking section based on the captured image by the in-vehicle camera functioning as the periphery detection device. It is preferable to set. If a road marking indicating a parking section is recognized as an image, a parking target can be set with high accuracy. Further, since the display image for instructing the target setting is made from the photographed image, it is easy to obtain consistency when recognizing the road marking from the photographed image. Moreover, a parking target can be set using only a vehicle-mounted camera without using other sensors such as sonar.

  In addition, the parking target setting unit of the parking assistance device according to the present invention determines a region of interest in the captured image based on the desired parking position on the display image set by the user by the second operation. It is preferable that the parking target is set by setting and recognizing the road marking in the region of interest. Image recognition is a process with a relatively heavy calculation load, but the calculation load is reduced by setting a region of interest and performing image recognition of a road marking in the region of interest.

  Further, the parking assist device according to the present invention includes an empty space detection unit that detects an empty space around the vehicle based on a measurement result of a distance sensor that measures a distance from the vehicle, and the parking target setting unit It is preferable that the empty space including the position corresponding to the desired parking position on the display image in the actual space is set as the parking target. By using a distance sensor such as a clearance sonar, an empty space around the vehicle can be easily detected. When a plurality of vacant spaces are detected by the vacant space detection unit, it is necessary to determine again in which vacant space the parking target is set. At this time, if the parking assistance device automatically selects, there is a possibility that a section different from the user's intention is set as the parking target. However, according to this configuration, an empty space including the position designated by the second operation is set as the parking target. That is, since the parking target is set at a place designated by the user, the convenience for the user is also improved.

  In addition, the display image providing unit of the parking assistance device according to the present invention indicates the parking target based on the parking target set by the parking target setting unit after the second operation. Change to a frame-shaped parking target icon. By changing the display form of the setting icon into a frame and becoming a parking target icon, it is clearly communicated to the user that the parking target has been set correctly at the location specified by the user. That is, the excellent graphic user interface gives the user a sense of security and improves convenience.

  In the parking assist apparatus according to the present invention, it is preferable that a plurality of the vehicle-mounted cameras are provided in the vehicle, and the display image includes an overhead image synthesized based on a plurality of captured images from the plurality of vehicle-mounted cameras. It is. A bird's-eye view image in the form of a map is easy to grasp the sense of distance between the vehicle and the parking area, and is easy for a user to perform a parking target setting instruction using a setting icon. In other words, the convenience of the user is improved by having an excellent graphic user interface.

A perspective view of a vehicle showing a partially cut-out vehicle Top view showing the shooting range of the camera Block diagram schematically showing an example of a system configuration of a vehicle Block diagram schematically showing an example of the functional configuration of the parking assistance device Diagram showing an example of the monitor screen Diagram showing an example of the monitor screen Diagram showing an example of the monitor screen Diagram showing an example of the monitor screen Diagram showing an example of the monitor screen The figure which shows an example of the path | route guided by a parking assistance apparatus Block diagram schematically showing another example of the functional configuration of the parking assistance device Explanatory drawing which shows the example which detects empty space Explanatory drawing which shows the example which detects empty space in a parking lot The figure which shows the example of the screen of the monitor apparatus in another embodiment typically The figure which shows the example of the screen of the monitor apparatus in another embodiment typically The figure which shows the other example of the path | route guided by a parking assistance apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the present embodiment, a parking support device (driving support device) that generates a bird's-eye view image in a form in which the vehicle is looked down from above is displayed on a monitor device based on images taken by a plurality of cameras provided in the vehicle. This will be described as an example. By using the bird's-eye view image, it is possible to provide driving assistance to the driver and monitoring assistance for obstacles around the vehicle. Hereinafter, the embodiment of the parking assistance device of the present invention will be described using an example of performing parking assistance during parking in a garage using an overhead image.

  A plurality of in-vehicle cameras 1 are installed in the vehicle 90. As shown in FIGS. 1 and 2, the rear portion of the vehicle 90, that is, the back door 91 is provided with a rear camera 1 a. A left side camera 1b is provided below the left side mirror 94 provided in the left front door 92, and a right side camera 1c is provided below the right side mirror 95 provided in the right front door 93. A front camera 1d is provided at the front of the vehicle 90. In the following description, these cameras 1a to 1d will be collectively referred to as a camera 1 (vehicle camera) as appropriate.

  The camera 1 uses a CCD (charge coupled device), CIS (CMOS image sensor), or other imaging device to capture a 15-30 frame per second two-dimensional image in time series, and digitally converts it into moving image data (captured image). Is a digital camera that outputs in real time. The camera 1 includes a wide angle lens. In particular, in the present embodiment, a viewing angle of 140 to 190 ° is ensured in the horizontal direction. The rear camera 1a and the front camera 1d have a depression angle of about 30 degrees on the optical axis and are installed in the vehicle 90, and can capture an area from the vehicle 90 to about 8 m. The left side camera 1b and the right side camera 1c are installed at the bottom of the side mirrors 94 and 95 with the optical axis facing downward, and a part of the side surface of the vehicle 90 and the road surface (ground) can be photographed.

  As shown in FIG. 3, an image photographed by the camera 1 can be displayed on the monitor device 4 via an image processing module 2 having a superimposing unit 2a, a graphic drawing unit 2b, a frame memory 2c, and the like. The two-dimensional image of each frame is stored in the frame memory 2c, and image processing or graphic superimposition can be performed for each frame. Further, the image processing module 2 combines the captured images captured by the plurality of cameras 1 to generate a combined image with a wider field of view, or generates a bird's-eye view image by converting the viewpoints of the captured image and the combined image. It is also possible to do.

  An overlapping area W (see FIG. 2) in the captured images of the two cameras is formed at the outer position of the front and rear corners of the vehicle 90, and image processing for making the boundary portion inconspicuous in the overlapping area W is performed. Thus, a composite image is generated. In addition, the viewpoint of the composite image is converted, and the appearance image (graphic image) of the roof of the vehicle 90 is superimposed on the image after the viewpoint conversion to generate an overhead image. The graphic image may be an illustration image that schematically or realistically represents the appearance of the roof of the vehicle 90, or may be a photographic image or a video image that actually captures the appearance of the roof of the vehicle 90. Needless to say, the viewpoints of the captured images of the single camera 1 may be converted, and the captured images of the plurality of cameras 1 whose viewpoints have been converted may be combined. Since image processing techniques related to the synthesis of a plurality of captured images and image viewpoint conversion are well known, detailed description thereof will be omitted. A drawing instruction to the graphic drawing unit 2b and a graphic superimposing instruction to the superimposing unit 2a are issued from a CPU (central processing unit) 5 described later.

  As the monitor device 4, for example, a monitor device of a navigation system is also used. As shown in FIG. 3, the monitor device 4 includes a display unit 4a, a touch panel 4b formed on the display unit 4a, and a speaker 4c. The display unit 4a displays a captured image of the camera 1 provided from the image processing module 2, a graphic image, a combined image obtained by combining them, and the like. As an example, the display unit 4a is configured by a liquid crystal display. The touch panel 4b is a pressure-sensitive or electrostatic instruction input device that is formed together with the display unit 4a and can output a contact position by a finger or the like as location data. FIG. 3 illustrates the case where the speaker 4c is provided in the monitor device 4, but the speaker 4c may be provided in another place such as the inside of the door. The speaker 4c outputs sound provided from the sound processing module 3 in accordance with an instruction from the CPU 5. Note that the CPU 5 may simply sound a notification sound via the buzzer 8.

  The CPU 5 performs advanced calculation processing such as image recognition and course prediction, and plays a central role in the parking assistance device 10. The CPU 5 executes various arithmetic processes using programs and parameters stored in the program memory 6. Further, the CPU 5 temporarily stores a captured image or the like in the work memory 7 as necessary, and executes the calculation. Here, an example in which the program memory 6 and the work memory 7 are memories different from the CPU 5 is shown, but they may be integrated in the same package as the CPU 5. The parking assistance device 10 is configured as a parking assistance ECU (electronic control unit) 9 together with a CPU 5, a memory, and other peripheral circuits. In this example, the CPU 5 is the core, but the parking assistance device 10 may be configured with another logic processor or logic circuit such as a DSP (digital signal processor) as the core.

The CPU 5 is communicably connected to various systems and sensors via an in-vehicle network denoted by reference numeral 50 in FIG. In the present embodiment, a CAN (controller area network) 50 is illustrated as an in-vehicle network. As shown in FIG. 1, the parking assist device 10 (CPU 5) is connected to a power steering system 31 and a brake system 37 in the vehicle. Each of these systems is configured with an electronic circuit such as a CPU as a core, similar to the parking support apparatus 10, and is configured with an ECU configured with peripheral circuits as a core, similar to the parking support ECU 9.

  The power steering system 31 is an electric power steering (EPS) system or an SBW (steer-by-wire) system. In this system, assist torque can be applied to the steering wheel operated by the driver by the actuator 41. It is also possible to perform automatic steering by driving the steering wheel or steering wheel by the actuator 41. The brake system 37 has an anti lock braking system (ABS) that suppresses the locking of the brake, a skid prevention device (ESC: electronic stability control) that suppresses the skidding of the vehicle during cornering, and a brake assist that enhances the braking force. Electric brake system and BBW (brake-by-wire) system. This system can apply a braking force to the vehicle 90 via the actuator 47.

  In FIG. 3, as an example of various sensors, a steering sensor 21, a wheel speed sensor 23, a shift lever switch 25, and an accelerator sensor 29 are connected to the CAN 50. The steering sensor 21 is a sensor that detects the steering amount (rotation angle) of the steering wheel, and is configured using, for example, a Hall element. The parking assist device 10 acquires the steering amount of the steering wheel by the driver and the steering amount at the time of automatic steering from the steering sensor 21 and executes various controls.

  The wheel speed sensor 23 is a sensor that detects the amount of rotation of the wheel of the vehicle 90 and the number of rotations per unit time, and is configured using, for example, a Hall element. The parking assistance device 10 calculates the amount of movement of the vehicle 90 based on the information acquired from the wheel speed sensor 23 and executes various controls. The wheel speed sensor 23 may be provided in the brake system 37. The brake system 37 performs various controls by detecting a brake lock, an idle rotation of the wheel, a sign of skidding, and the like from the difference in rotation between the left and right wheels. When the wheel speed sensor 23 is provided in the brake system 37, the parking assistance device 10 acquires information via the brake system 37. The brake sensor 27 is a sensor that detects the amount of operation of the brake pedal, and the parking assistance device 10 acquires information via the brake system 37. For example, when the brake pedal is depressed during automatic steering, the parking assist device 10 can perform control to interrupt or stop automatic steering because it is in an unfavorable environment for automatic steering.

  The shift lever switch 25 is a sensor or switch that detects the position of the shift lever, and is configured using a displacement sensor or the like. For example, when the shift is set to reverse, the parking assist device 10 can start assist control, or can end assist control when the shift is changed from reverse to forward. Further, the torque sensor 22 that detects the operation torque to the steering wheel can detect whether or not the driver is holding the steering wheel. The parking assist device 10 performs control to interrupt or stop the automatic steering because it is in an unfavorable environment for the automatic steering when the driver grips the steering wheel strongly during the automatic steering. Can do. Further, during automatic steering, creeping of the vehicle 90 by engine idling is generally used. Therefore, when it is detected by the accelerator sensor 29 that the driver has operated the accelerator, the parking assistance device 10 can perform control to interrupt or stop the automatic steering because it is in an environment unfavorable for automatic steering. it can.

  The various systems and sensors shown in FIG. 3 and their connection forms are examples, and other configurations and connection forms may be employed. Further, as described above, the sensor may be directly connected to the CAN 50 or may be connected via various systems.

As described above, the parking assistance device 10 is configured with the CPU 5 as a core, and performs various calculations for parking assistance in cooperation with a program (software) stored in the program memory 6. As a kind of parking assistance,
(1) Projecting an image of the rear of the vehicle on a monitor mounted in the vehicle and superimposing guide lines such as a vehicle width extension line and an expected course line,
(2) A parking target is set and a driver's operation is instructed by voice, etc.
(3) Furthermore, there are drivers where the driver is only responsible for speed adjustment and is guided to the parking target by automatic steering. In (2) and (3), a parking target is set. There are various methods for setting the parking target. For example,
(A) When passing through the parking target position, a vacant area is detected with a sonar (clearance sonar 33) and automatically recognized to detect a parking target,
(B) For example, detecting a parking target by recognizing a marking line,
(C) A combination of a plurality of methods such as (a) and (b) above for improving accuracy,
and so on.

  In the present embodiment, as shown in (3) above, a parking assistance device that guides to a parking target by automatic steering is illustrated, and although details will be described later, the setting of the parking target is shown in (b) above. Thus, the case where the image of the lane marking is recognized is illustrated.

  As shown in FIG. 4, the parking assist device 10 includes a display image providing unit 11, a target setting instruction receiving unit 13, a parking target setting unit 15, a route calculation unit 17, and an automatic steering control unit 19. Is provided. Each functional unit is realized by cooperation of hardware and software configured as the driving support ECU 9.

  The display image providing unit 11 is a functional unit that provides a display image including a captured image captured by the camera 1 and a touch button (setting button) superimposed on the captured image to the monitor device 4 with the touch panel 4b. The target setting instruction receiving unit 13 is a functional unit that receives a parking target setting instruction on the touch panel 4b by a user such as a driver. On the touch panel 4b, a setting icon for setting a parking target is displayed as a touch button. The target setting instruction accepting unit 13 detects the user's first operation on the setting icon and is an operation following the first operation. The user sets the desired parking position on the display image displayed on the monitor device 4. The second operation is detected and a parking target setting instruction is received.

  The parking target setting unit 15 is a functional unit that sets a parking target using a detection result of a surrounding detection device that detects a situation around the vehicle 90. In the present embodiment, the periphery detection device is the camera 1, and as shown in (b) above, the parking target setting unit 15 recognizes the road marking indicating the parking area based on the captured image. Set parking targets. The route calculation unit 17 calculates a guidance route until the vehicle 90 reaches the parking target based on the parking target. The automatic steering control unit 19 controls the power steering system 31 based on the guidance route, and guides the vehicle 90 to the parking target by automatic steering.

  Hereinafter, description will be given using screen examples of the monitor device 4 shown in FIGS. Here, a multi-view screen is shown in which a bird's-eye view image obtained by combining images taken by a plurality of cameras 1 and a shot image of any single camera 1 are displayed side by side, but only the bird's-eye view image is displayed. May be. In the bird's-eye view image, the host vehicle 90 is not displayed as an image captured by the camera 1 but as an icon J based on a previously captured image or a drawn image. The code | symbol 100 in FIGS. 5-9 is the parked vehicle 100 already parked.

  As one aspect, as shown in FIGS. 5 and 6, the first operation is a drag operation that touches the setting icon P, continues the touch, and moves the setting icon P on the display image. Further, as shown in FIGS. 6 and 7, the second operation is a drop operation for releasing the setting icon P that has been moved by the drag operation at a desired parking position on the display image. Here, the desired parking position is an approximate position where the user wants to park the vehicle 90, and is, for example, a parking section E indicated by a road marking Y such as a white line. The user moves the setting icon P into the parking area E displayed on the monitor device 4 and opens (drops) the setting icon P in the parking area E. That is, as one aspect, a drag and drop operation can be applied.

  As another aspect, the first operation is a touch operation on the setting icon P, and the second operation is a touch operation on a desired parking position on the display image within a predetermined time from the first operation. Also good. That is, without dragging, the user may move the setting icon P into the parking area E displayed on the monitor device 4 by two touch operations within a predetermined time. That is, as an aspect, a touch and touch operation can be applied.

  In FIG. 5 to FIG. 6, the setting icon P is operated by the user's finger, but this does not prevent the operation using another pointing device such as a mouse or a joystick. That is, the operation means and the input means for performing the operation are not limited to those directly giving instructions to the touch panel 4b such as a user's finger, but instructions are given via a pointing device such as a mouse and the display control unit 2. Also includes what you give. When another pointing device such as a mouse or a joystick is used, a pointer indicating the operation position or operation state is displayed on the screen, so that the same operation as that performed by a finger is possible.

  As described above, in the present embodiment, the parking target setting unit 15 recognizes the road marking Y indicating the parking section E based on the captured image by the in-vehicle camera 1 functioning as the surrounding detection device, and the process shown in FIG. A parking target T as shown is set. At this time, the parking target setting unit 15 sets a region of interest ROI as an image processing target region as shown in FIG. 8 in order to reduce the calculation load of image recognition. The parking target setting unit 15 sets the parking target T by recognizing the road marking Y in the region of interest ROI. Specifically, the region of interest ROI in the captured image is set based on the desired parking position on the display image set by the user by the second operation. Here, the captured images may be images captured by a plurality of cameras 1, may be a plurality of images, or may be a combined bird's-eye view image.

  In general, since the road surface is a dark color such as asphalt and the road surface marking Y is a light color such as white or yellow, the contrast of the road surface marking Y is high. Therefore, the road marking Y can be easily extracted by known edge detection. And in real space, since many parts of the road marking Y are formed with the straight line, the road marking Y is a straight line in the bird's-eye view image. Therefore, the parking target setting unit 15 can recognize the road marking Y as an image by using a known Hough conversion, RANSAC (random sample consensus), or the like. When recognizing the road marking Y from the captured image of the camera 1 itself, there is a possibility that the image is distorted. However, the road marking Y can be recognized by using a known image processing technique such as image recognition after correcting the distortion or the like, or matching a straight line in consideration of the distortion or the like.

  After the second operation, the display image providing unit 11 changes the setting icon P to a frame-shaped parking target icon F indicating the parking target based on the parking target T set by the parking target setting unit 15. Thereby, the user can easily recognize that the setting of the parking target T has been completed, and the parking position can be confirmed by the frame-shaped parking target icon F. Moreover, it is preferable to switch the display of the captured image on the multi-view screen to the captured image of the camera 1 in the direction in which the parking target T exists so that the parking target T can be confirmed in detail. In the present embodiment, the image is switched to a captured image of the left side camera 1b.

  When the parking target T is set, the route calculation unit 17 calculates the guidance route K of the vehicle 90 as shown in FIG. In the present embodiment, since the parking target T is next to the stop position of the vehicle 90, the guidance route K that once moves forward to the reverse start position, moves backward from the reverse start position, and reaches the parking target T is derived. . The automatic steering control unit 19 controls the power steering system 31 based on the guidance route K, and guides the vehicle 90 to the parking target by automatic steering. In this guide route K, it is necessary to switch the shift lever from forward to backward at the reverse start position, which is performed manually by a driver. In addition, the driver operates the brakes such as starting from the position where the vehicle is parked to set the parking target T, stopping at the reverse start position, starting from the reverse start position, and stopping when the parking target T is reached. Is done.

  This will be specifically described below. When the parking target T is set and the guidance route K is calculated, the parking assist device 10 “starts automatic steering. Touch the steering lightly and loosen the brake.” A voice message such as When the vehicle 90 reaches the reverse start position, the parking assist device 10 issues a voice message “Please apply the brake and stop”. Here, the automatic steering control unit 19 may automatically stop the vehicle 90 via the brake system 37. When it is detected by the brake sensor 27 that the vehicle 90 has stopped at the reverse start position, the parking assist device 10 issues a voice message “Please change the shift lever to reverse.” The parking assist device 10 then issues a voice message such as “Start automatic steering. Touch the steering wheel lightly and loosen the brake.” When the vehicle 90 falls within the parking target T, the parking assistance device 10 issues a voice message “Please apply the brake to stop. Parking assistance will be terminated.” Here, the automatic steering control unit 19 may automatically stop the vehicle 90 via the brake system 37, and a voice message “parking support is ended” may be issued.

  As described above, the vehicle 90 includes sensors that detect the behavior of the vehicle 90, such as the wheel speed sensor 23 and the steering sensor 21. The parking assistance device 10 can automatically steer the vehicle 90 by a known dead-reckoning technique or the like based on the detection results of these sensors that detect the movement state of the vehicle 90.

  In addition, in this embodiment, although the form in which automatic steering is implemented was illustrated, it does not prevent that driving operation including steering is performed manually. After the guide route K is calculated, the operation amount and the operation direction of the steering wheel may be guided, and a guide image such as an expected trajectory line or a target line may be superimposed on the monitor device 4 to support driving by manual operation. .

[Another embodiment]
In the above embodiment, as shown in the above (b), the case where the image of the lane marking is recognized when setting the parking target T has been described as an example. However, as shown in (a) above, an empty area may be detected by a sonar (clearance sonar 33) or the like. FIG. 11 schematically illustrates an example of a functional configuration of the parking assistance device 10 according to another embodiment. Functional blocks indicated by the same reference numerals as those in FIG. 4 are the same as those in the above-described embodiment. In FIG. 11, an empty space detection unit 12 that detects an empty space around the vehicle 90 based on a measurement result of a distance sensor (periphery detection device) that measures the distance from the vehicle 90 is provided. Here, the clearance sonar 33 is used as a distance sensor corresponding to the periphery detection device, and an empty space is detected based on the detection result. The parking target setting unit 15 sets an empty space including a position corresponding to a desired parking position on the display image in the actual space as a parking target.

  As shown in FIG. 12, a clearance sonar 33 (distance sensor) as a point sensor is mounted on the vehicle 90 toward the side. Other distance sensors such as a single beam sensor and a laser radar may be mounted. When the vehicle 90 passes by another parked vehicle 100 (hereinafter referred to as a parked vehicle), the distance to the parked vehicle 100 is measured by the clearance sonar 33. For example, the clearance sonar 33 measures the distance from the parked vehicle 100 according to the movement of the vehicle 90 and acquires the surface shape information. The surface shape information of the parked vehicle 100 is discrete data corresponding to the moving distance of the vehicle 90. The moving speed, moving distance, and moving time of the vehicle 90 are determined linearly. For example, when the vehicle 90 moves at a constant speed, if it is measured according to a predetermined time interval, it is measured according to the moving distance. The ECU 9 can earn the moving speed and the moving distance of the vehicle 90 based on the detection result of the wheel speed sensor 23. Any method may be used as long as the surface shape information can be obtained almost uniformly as a result.

  The free space detection unit 12 calculates the degree of matching between the bumper shape of a general vehicle stored in the program memory 6 and the surface shape information. Then, when it is determined that the surface shape information corresponds to the bumper shape of the vehicle according to a predetermined standard, an area where the surface shape information exists is detected as “already parked space”. On the other hand, an area that does not correspond to “already parked space” is detected as “empty space”.

  FIG. 5 is an overhead view showing an example of detecting “empty space” while the vehicle 90 passes through a passage of a parking lot such as a shopping center. A space where no parked vehicle 100 exists, for example, a space between two parked vehicles 100 parked in the adjacent parking section, is also detected as an empty space in principle. However, when a predetermined length along the traveling direction of the vehicle 90 in FIG. 5 is set as a threshold value, and it is detected that the space is not a parking space for a length longer than the threshold value, the space is an empty space. Detected as Further, when it is detected that adjacent parking sections are not already parked spaces, in principle, it is detected as an empty space for one vehicle. However, after detecting such a wide empty space, the empty space may be divided and provided with virtual boundaries to be set as a plurality of empty spaces. Since the vacant space is detected as the vehicle 90 moves, all the vacant space existing within a predetermined range from the vehicle 90 is accumulated in the accumulation unit having the work memory 7 as a core.

  The empty space may be detected with higher accuracy by recognizing the road marking Y in the empty space as described above. A reference symbol V in FIG. 13 indicates a photographing range of the rear camera 1 a mounted on the vehicle 90. Since the vehicle 90 moves and detects an empty space using a distance sensor, the shooting range V of the vehicle 90 also changes as the vehicle 90 moves. Therefore, it is preferable to recognize the road marking Y sequentially for the empty space detected as the vehicle 90 moves to detect the empty space.

  In the present embodiment, an example will be described in which the parking target T is set on the monitor 4 on which a captured image captured by a single camera 1 is displayed instead of an overhead image. FIG. 14 shows an example in which an image captured by the rear camera 1a of the vehicle 90 at the position shown in FIG. Since the monitor device 4 is disposed in front of the seating position of the driver, the image displayed on the display unit 4a of the monitor device 4 is obtained by reversing the left and right mirror images of the captured image as shown in FIG. Become. That is, it is displayed so that the driver can obtain the same visual effect as when viewing the scene behind the vehicle 90 through the rearview mirror. In FIG. 14, the empty space G detected by the empty space detector 12 is indicated by a broken line, but this need not be actually displayed. Of course, the display may allow the user to select a desired parking section E from the empty space G.

  As described above with reference to FIGS. 5 and 6, the user touches the setting icon P and performs a drag operation to continue the touch and move the setting icon P on the display image. Then, as described above with reference to FIGS. 6 and 7, a drop operation is performed to release the setting icon P that has been moved by the drag operation at a desired parking position on the display image. As described above with reference to FIG. 9, the display image providing unit 11 displays the setting icon P as a parking target based on the parking target T set by the parking target setting unit 15 after the second operation. Change to the parking target icon F. Thereby, the user can easily recognize that the setting of the parking target T has been completed, and the parking position can be confirmed by the frame-shaped parking target icon F.

  Here, the case where the first operation and the second operation are drag and drop is illustrated, but as described above, touch and touch may be used. In addition, here, the monitor device 4 is a monitor device with the touch panel 4b and the setting button displayed as the setting icon P is a touch button. However, the monitor device 4 does not include the touch panel 4b. May be. That is, it is not always necessary to directly touch the setting button displayed on the display unit 4a with a user's finger or the like, and the parking target T may be set using a pointing device. .

  In the case of drag-and-drop, the first operation is a drag operation that performs a designation operation that designates the setting icon P as an operation target, continues the designation operation, and moves the setting icon P on the display image. The second operation may be a drop operation that opens the setting icon P that has been moved by a drag operation at a desired parking position on the display image. In the case of touch-and-touch, the first operation is a designation operation that designates the setting icon P as an operation target, and the second operation is performed on the display image within a predetermined time from the first operation. The designation operation may designate a desired parking position. Here, for example, when the pointing device is a mouse or a joystick, this designation operation is preferably a click operation.

  Also in this separate embodiment, when the parking target T is set, the route calculation unit 17 calculates the guidance route K of the vehicle 90. In the present embodiment, since the parking target T is not behind the stop position of the vehicle 90 but behind it, as shown in FIG. 16, a guidance route K that once goes backward without reaching forward and reaches the parking target T is derived. Is done. Of course, even if it is behind, when it cannot reverse and park, the guidance path | route K accompanied by advancing is calculated as demonstrated using FIG. When the automatic steering control unit 19 is provided, the automatic steering control unit 19 controls the power steering system 31 based on the guidance route K and guides the vehicle 90 to the parking target by automatic steering. Dead reckoning, automatic braking, voice messages for guidance, and the like are the same as those in the above-described embodiment, and those skilled in the art can easily understand the differences, so detailed description thereof is omitted. Further, in the present embodiment, the embodiment in which the automatic steering is performed is exemplified, but this does not prevent the driving operation including the steering from being performed manually. After the guide route K is calculated, the operation amount and the operation direction of the steering wheel may be guided, and a guide image such as an expected trajectory line or a target line may be superimposed on the monitor device 4 to support driving by manual operation. .

  As described above, according to the present invention, a parking target for parking assistance can be easily and accurately set with an operation by a user.

1: Camera (vehicle-mounted camera, peripheral detection device)
1a: Rear camera (vehicle camera)
1b: Left side camera (vehicle camera)
1c: Right side camera (vehicle camera)
1d: Front camera (vehicle camera)
4: Monitor device 11: Display image providing unit 12: Empty space detecting unit 13: Target setting instruction receiving unit 15: Parking target setting unit 33: Clearance sonar (distance sensor)
E: Parking area (desired parking position)
F: Parking target icon G: Empty space P: Setting icon ROI: Region of interest T: Parking target Y: Road marking

Claims (9)

A display image providing unit that provides a monitor image with a display image including a captured image captured by the in-vehicle camera and a setting button superimposed on the captured image;
On the display image displayed on the monitor device, the first operation of the user with respect to the setting icon displayed as the setting button for detecting a parking target is detected and the operation is subsequent to the first operation. A target setting instruction receiving unit that detects a second operation of the user with respect to a desired parking position and receives a setting instruction of the parking target by the user;
A space in which the vehicle can be parked in a region in an actual space corresponding to the desired parking position on the display image is used as the parking target using a detection result of a surrounding detection device that detects a situation around the vehicle. A parking assist device comprising: a parking target setting unit to be set.   The first operation is a drag operation for performing a designation operation for designating the setting icon as an operation target, continuing the designation operation and moving the setting icon on the display image, and the second operation is performed. The parking assist device according to claim 1, wherein the parking assist device is a drop operation for releasing the setting icon moved by the drag operation at the desired parking position on the display image.   The first operation is a designation operation that designates the setting icon as an operation target, and the second operation designates the desired parking position on the display image within a predetermined time from the first operation. The parking assist device according to claim 1, wherein the parking assist device is a designation operation to be performed.   The monitor device according to claim 2 or 3, wherein the monitor device is a monitor device with a touch panel, the setting button displayed as the setting icon is a touch button, and the designation operation is a touch operation by the user. Parking assistance device.   5. The parking target setting unit according to claim 1, wherein the parking target setting unit sets the parking target by recognizing a road marking indicating a parking area based on the captured image by the in-vehicle camera functioning as the periphery detection device. The parking assistance device according to one item.   The parking target setting unit sets a region of interest in the captured image based on the desired parking position on the display image set by the user by the second operation, and the region of interest includes the region of interest. The parking assistance device according to claim 5, wherein the parking target is set by recognizing a road marking.   An empty space detection unit that detects an empty space around the vehicle based on a measurement result of a distance sensor that measures a distance from the vehicle, and the parking target setting unit is displayed on the display image in an actual space. The parking assistance device according to any one of claims 1 to 4, wherein the empty space including a position corresponding to the desired parking position is set as the parking target.   The display image providing unit changes the setting icon to a frame-shaped parking target icon indicating the parking target based on the parking target set by the parking target setting unit after the second operation. The parking assistance device according to any one of Items 1 to 7.   The said vehicle-mounted camera is provided with two or more in the said vehicle, The said display image contains the bird's-eye view image synthesize | combined based on the some picked-up image from these vehicle-mounted cameras. Parking assistance device.

Images