JP7351172B2 - parking assist device - Google Patents

Description

Embodiments of the present invention relate to a parking assistance device.

2. Description of the Related Art Parking support devices that guide a vehicle to a parking target position in a parking space (for example, a home garage) have been put into practical use. For example, by comparing information about the surroundings of the parking target position that has been learned in advance with information about the surroundings of the parking target position acquired when actually parking, and identifying the own vehicle position, the information can be used in the past. There is a method that guides a vehicle to a parking space by simply calculating the difference in the travel route.

Special Publication No. 2013-530867

However, in a home garage or the like, the surrounding situation around the parking target position may change. For example, new shelves may be installed or items may be regularly placed on the shelves. In such cases, there is a problem that the surrounding information that has been learned in advance and the surrounding information obtained when actually parking is not consistent, and it may not be possible to identify the vehicle's location. was there. As a result, parking assistance processing may be canceled even if a travel route that has been used in the past is available. Therefore, it would be meaningful if a parking assistance device could be provided that can respond to changes in the surrounding conditions of the parking target position and that can continuously identify the vehicle's position with high accuracy.

For example, in the case where a vehicle is repeatedly parked at the same parking target position, the parking assistance device according to the embodiment of the present invention detects a surrounding area including the parking target position detected by a detection unit provided in the vehicle. a situation information acquisition unit that obtains surrounding situation information; first situation information indicating the surrounding situation information that has been obtained when the vehicle completes parking at the parking target position; and a storage control unit that stores the travel route up to, and compares the first situation information with second situation information indicating the surrounding situation information acquired when the vehicle newly parks at the parking target position. Then , the above comparison is performed to determine whether new information not included in the first situation information is included in the second situation information and whether there is interference between the area specified by the new information and the area where the movement route exists. If the area specified by the new information does not interfere with the area where the movement route exists , the area specified by the new information is determined based on the result of an information update unit that updates first situation information; a current position estimation unit that estimates the current position of the vehicle by comparing the first situation information and the second situation information; and a guidance control unit that guides the vehicle to the parking target position according to the travel route. According to this configuration, for example, even if the surrounding situation of the parking target position changes, the current guidance process can be continued as long as there is no interference between the area specified by the new information and the area where the movement route exists. Next time, it will be possible to estimate the vehicle's current position with high accuracy based on the latest surrounding situation information. As a result, guidance processing using the stored travel route can be continued with high accuracy.

The information updating unit of the parking assistance device according to the embodiment of the present invention extracts, for example, an object boundary line that can be considered as a boundary line between the road surface and the object included in the first situation information and the second situation information, and The first situation information and the second situation information may be compared. According to this configuration, for example, it is possible to easily determine whether there is interference between the area specified by the new information and the movement route.

The information update unit of the parking assistance device according to the embodiment of the present invention, for example, extracts difference information between the first situation information and the second situation information, and updates the first situation information using the difference information. You may also do so. According to this configuration, the first status information update process when new information is extracted can be performed with a minimum processing load.

FIG. 1 is a schematic plan view showing an example of a vehicle that can be equipped with a parking assistance device according to an embodiment. FIG. 2 is an exemplary block diagram of the configuration of a control system including the parking assist device according to the embodiment. FIG. 3 is a block diagram exemplarily showing the configuration of the parking assistance device (parking assistance unit) according to the embodiment when implemented by a CPU. FIG. 4 shows a three-dimensional space diagram showing feature points of objects based on first situation information, a parking target position, and a movement route, and map data based on first situation information in the parking assistance device according to the embodiment. FIG. 2 is an explanatory diagram illustratively and schematically shown. FIG. 5 shows a three-dimensional space diagram showing feature points of an object, a parking target position, and a movement route based on the second situation information, and an update based on the second situation information in the parking assistance device according to the embodiment. FIG. 2 is an explanatory diagram illustratively and schematically showing map data before being processed. FIG. 6 is a three-dimensional space diagram showing object boundaries, feature points, parking target positions, and travel routes based on a comparison between first situation information and second situation information in the parking assistance device according to the embodiment. , is an explanatory diagram exemplarily and schematically showing map data in which object boundaries are taken into consideration. FIG. 7 is an explanatory diagram exemplarily and schematically showing the map data after the update based on the second situation information is executed in the parking assistance device according to the embodiment. FIG. 8 is a flowchart illustrating an example of the flow of parking assistance processing by the parking assistance device according to the embodiment.

Exemplary embodiments of the invention are disclosed below. The configuration of the embodiment shown below, and the actions, results, and effects brought about by the configuration are examples. The present invention can be realized with configurations other than those disclosed in the embodiments below, and it is possible to obtain at least one of various effects based on the basic configuration and derivative effects. .

FIG. 1 is a schematic plan view of a vehicle 1 in which a parking assist device of this embodiment is mounted. The vehicle 1 may be, for example, an automobile (internal combustion engine automobile) whose driving source is an internal combustion engine (engine, not shown), or an automobile (electric automobile) whose driving source is an electric motor (motor, not shown). , fuel cell vehicle, etc.), or a vehicle (hybrid vehicle) that uses both of these as drive sources. Further, the vehicle 1 can be equipped with various transmission devices, and can be equipped with various devices (systems, parts, etc.) necessary for driving an internal combustion engine or an electric motor. Further, the system, number, layout, etc. of devices related to driving the wheels 3 (front wheels 3F, rear wheels 3R) in the vehicle 1 can be set in various ways.

As illustrated in FIG. 1, the body 2 of the vehicle 1 is provided with, for example, a plurality of imaging units 15, a plurality of radars 16, a sonar 17, etc. as detection units that acquire situation information around the vehicle 1. ing.

As the plurality of imaging units 15, for example, four imaging units 15a to 15d are provided. The imaging unit 15 is, for example, a digital camera incorporating an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS image sensor). The imaging unit 15 can output moving image data (captured image data) at a predetermined frame rate. The imaging units 15 each have a wide-angle lens or a fisheye lens, and can image a range of, for example, 140° to 220° in the horizontal direction. Furthermore, the optical axis of the imaging unit 15 (15a to 15d) may be set to point diagonally downward, for example. Therefore, the imaging unit 15 (15a to 15d) detects road surfaces on which the vehicle 1 can move, marks attached to the road surfaces (arrows, division lines, parking frames indicating parking spaces, lane separation lines, etc.) and objects (as obstacles). , pedestrians, other vehicles, etc.) are sequentially photographed and output as captured image data.

The imaging unit 15a is located, for example, at the rear end 2e of the vehicle body 2, provided on the wall below the rear window, and images the situation in the rear region of the vehicle 1. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2, is provided on the right side door mirror 2g, and images the situation in an area including the right front, right side, and right rear of the vehicle 1. The imaging unit 15c is located, for example, at the front side of the vehicle body 2, that is, at the front end 2c in the vehicle longitudinal direction, is provided on a front bumper, etc., and images the situation in the front area of the vehicle 1. The imaging unit 15d is located, for example, on the left side of the vehicle body 2, that is, on the left end 2d in the vehicle width direction, and is provided on the door mirror 2g as a left side protrusion, and is provided on the left front, left side, and left rear of the vehicle 1. image the situation in the area including the area. The ECU 14 (see FIG. 2) constituting the parking assist device executes arithmetic processing and image processing based on the captured image data obtained by the plurality of imaging units 15, and generates an image with a wider viewing angle, and It is also possible to generate a virtual overhead image of 1 viewed from above.

The vehicle 1 also includes a plurality of radars 16 as distance measuring units capable of measuring distances to objects existing outside the vehicle 1. The radar 16 is, for example, a millimeter wave radar or the like, and is capable of measuring the distance to an object existing in the direction in which the vehicle 1 is traveling. In this embodiment, the vehicle 1 has a plurality of radars 16a to 16d. The radar 16a is provided, for example, at the left end of the rear bumper of the vehicle 1, and is capable of measuring the distance to an object present on the left rear of the vehicle 1. Further, the radar 16b is provided at the right end of the rear bumper of the vehicle 1, and is capable of measuring the distance to an object present on the right rear side of the vehicle 1. The radar 16c is provided at the right end of the front bumper of the vehicle 1, and is capable of measuring the distance to an object present on the right front of the vehicle 1. Further, the radar 16d is provided at the left end of the front bumper of the vehicle 1, and is capable of measuring the distance to an object present on the left front of the vehicle 1.

The vehicle 1 also includes a sonar 17 that can measure the distance to an external object that is relatively close to the vehicle 1 using ultrasonic waves. In this embodiment, the vehicle 1 has a plurality of sonars 17a to 17h. The sonar 17a to 17d are provided on the rear bumper of the vehicle 1, and are capable of measuring the distance to an object existing behind the vehicle. The sonar 17e to 17h are provided on the front bumper of the vehicle 1, and are capable of measuring the distance to an object present in front of the vehicle 1.

FIG. 2 is an exemplary and schematic block diagram showing the functional configuration of the control system of the vehicle 1 including the parking assist device according to the present embodiment. As illustrated in FIG. 2, the control system includes an ECU 14, a radar 16, a sonar 17, etc., as well as a monitor device 11, a steering system 13, a brake system 18, a steering angle sensor 19, an accelerator sensor 20, a shift sensor 21, and wheels. A speed sensor 22, a drive system 24, and the like are electrically connected via an in-vehicle network 23 as a telecommunications line. The in-vehicle network 23 is configured as, for example, a CAN (controller area network). The ECU 14 can control the steering system 13, the brake system 18, the drive system 24, etc. by sending control signals through the in-vehicle network 23.

A display device 8 and an audio output device 9 as an audio output section are provided in the cabin of the vehicle 1. The display device 8 is, for example, an LCD (liquid crystal display), an OELD (organic electroluminescent display), or the like. The audio output device 9 is, for example, a speaker. Further, the display device 8 is covered with a transparent operation input section 10 such as a touch panel. The passenger (user) can visually recognize the image displayed on the display screen of the display device 8 via the operation input unit 10. Further, the occupant can perform operation input by touching, pressing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8. . The display device 8, the audio output device 9, the operation input unit 10, and the like are provided, for example, in a monitor device 11 located at the center of the dashboard in the vehicle width direction, that is, in the left-right direction. The monitor device 11 can have an operation input unit (not shown) such as a switch, a dial, a joystick, a push button, or the like. Further, an audio output device (not shown) can be provided at a different position in the vehicle interior than the monitor device 11, and audio can be output from the audio output device 9 of the monitor device 11 and other audio output devices. can. Note that the monitor device 11 may be used also as a navigation system or an audio system, for example.

The ECU 14 also receives the detection results of the torque sensor 13b, brake sensor 18b, steering angle sensor 19, radar 16, sonar 17, accelerator sensor 20, shift sensor 21, wheel speed sensor 22, etc. via the in-vehicle network 23, and the operation It is possible to receive operation signals and the like from the input unit 10.

The steering system 13 is an electric power steering system, an SBW (Steer By Wire) system, or the like. Steering system 13 includes an actuator 13a and a torque sensor 13b. The steering system 13 is electrically controlled by the ECU 14 or the like, and operates the actuator 13a to add torque to the steering unit (steering wheel, etc.) to supplement the steering force, thereby turning the wheels 3. steer. The torque sensor 13b detects the torque that the driver applies to the steering section, and transmits the detection result to the ECU 14.

The brake system 18 includes an ABS (Anti-lock Brake System) that controls the locking of the brakes of the vehicle 1, an electronic stability control (ESC) that suppresses skidding of the vehicle 1 during cornering, and an electronic stability control (ESC) that increases the braking force. It includes an electric brake system that assists the brakes and BBW (Brake By Wire). Brake system 18 includes an actuator 18a and a brake sensor 18b. The brake system 18 is electrically controlled by the ECU 14 and the like, and applies braking force to the wheels 3 via the actuator 18a. The brake system 18 detects signs of brake locking, idling of the wheels 3, skidding, etc. from the difference in rotation between the left and right wheels 3, and performs control to suppress locking of the brakes, idling of the wheels 3, and skidding. Execute. The brake sensor 18b is a displacement sensor that detects the position of a brake pedal as a movable part of the brake operation section, and transmits the detection result of the position of the brake pedal to the ECU 14.

The steering angle sensor 19 is a sensor that detects the amount of steering of a steering unit such as a steering wheel. The steering angle sensor 19 is composed of a Hall element or the like, detects the rotation angle of the rotating portion of the steering section as a steering amount, and transmits the detection result to the ECU 14. The accelerator sensor 20 is a displacement sensor that detects the position of an accelerator pedal as a movable part of the acceleration operation section, and transmits the detection result to the ECU 14.

The shift sensor 21 is a sensor that detects the position of a movable part (bar, arm, button, etc.) of the speed change operation section, and transmits the detection result to the ECU 14. The wheel speed sensor 22 is a sensor that includes a Hall element and the like and detects the amount of rotation of the wheel 3 and the number of rotations of the wheel 3 per unit time, and transmits the detection result to the ECU 14.

The drive system 24 is an internal combustion engine system or a motor system as a drive source. The drive system 24 controls the fuel injection amount and intake amount of the engine and the output value of the motor according to the amount of operation requested by the driver (user) (for example, the amount of depression of the accelerator pedal) detected by the accelerator sensor 20. . Moreover, the output values of the engine and motor can be controlled in cooperation with the control of the steering system 13 and the brake system 18, depending on the running state of the vehicle 1, regardless of the user's operation. For example, driving support such as normal driving support or parking support can be performed.

Note that the configuration, arrangement, electrical connection form, etc. of the various sensors and actuators described above are merely examples, and can be variously set (changed).

The ECU 14 is composed of a computer or the like, and controls overall control of the vehicle 1 through cooperation of hardware and software. Specifically, the ECU 14 includes a CPU (Central Processing Unit) 14a, a ROM (Read Only Memory) 14b, a RAM (Random Access Memory) 14c, a display control section 14d, an audio control section 14e, and an SSD (Solid State Drive) 14f. Equipped with The CPU 14a, ROM 14b, and RAM 14c may be provided within the same circuit board.

The CPU 14a can read a program installed and stored in a non-volatile storage device such as a ROM 14b, and perform arithmetic processing according to the program. The CPU 14a can execute a process of recognizing the surrounding situation of the vehicle 1 and the parking target position, for example, based on the surrounding situation information acquired by the imaging unit 15, the radar 16, the sonar 17, etc. In addition, the CPU 14a performs a process of extracting changes in the surrounding situation based on information indicating the surrounding situation of the vehicle 1 acquired at different timings (surrounding situation information), and estimates the current position of the vehicle 1 based on the surrounding situation information. Processing to store information that reflects changes in surrounding conditions, processing that utilizes stored information, processing that guides the vehicle 1 to the parking target position, etc. can be executed. In addition, when displaying the image captured by the imaging unit 15 on the display device 8, the CPU 14a performs arithmetic processing and image processing on the captured image (curved image) of the wide-angle image obtained by the imaging unit 15 to correct distortion. or generate an overhead image (surrounding image) in which a vehicle image (self-vehicle icon) representing the vehicle 1 is displayed at the center position, for example, based on the captured image captured by the imaging unit 15. I can do it. Furthermore, when generating the bird's-eye view image, the CPU 14a can change the position of the virtual viewpoint to generate a bird's-eye view image that looks at the vehicle image from directly above or an overhead image that looks at the vehicle image from an oblique direction. .

The ROM 14b stores various programs and parameters necessary for executing the programs. The RAM 14c temporarily stores various data used in calculations by the CPU 14a. Among the arithmetic processing performed by the ECU 14, the display control unit 14d mainly processes image data acquired from the imaging unit 15 and output to the CPU 14a, and performs image processing for displaying image data acquired from the CPU 14a on the display device 8. Executes conversion to data, etc. Among the calculation processing performed by the ECU 14, the audio control unit 14e mainly executes audio processing that is acquired from the CPU 14a and output to the audio output device 9. The SSD 14f is a rewritable nonvolatile storage unit that continues to store data acquired from the CPU 14a even when the ECU 14 is powered off. Note that the CPU 14a, ROM 14b, RAM 14c, etc. can be integrated in the same package. Further, the ECU 14 may have a configuration in which other logic processors such as a DSP (digital signal processor), logic circuits, etc. are used instead of the CPU 14a. Further, an HDD (hard disk drive) may be provided in place of the SSD 14f, or the SSD 14f and the HDD may be provided separately from the ECU 14.

FIG. 3 is a block diagram illustratively and schematically showing a configuration when the parking assist device (parking assist unit 30) according to the embodiment is implemented by the CPU 14a. By executing the parking assistance program read out from the ROM 14b, the CPU 14a, as shown in FIG. A parking support unit 30 including modules such as a position estimation unit 42 and a guidance control unit 44 is realized. Further, the situation information acquisition section 32 includes detailed modules such as an image information acquisition section 32a and a distance information acquisition section 32b. The storage control unit 38 includes detailed modules such as a situation information storage control unit 38a and a route storage control unit 38b. The information update section 40 includes detailed modules such as a new information extraction section 40a, an area determination section 40b, and an update processing section 40c. Further, the guidance control section 44 includes detailed modules such as a route validity determination section 44a and a guidance processing section 44b. Note that some or all of the situation information acquisition unit 32, mode switching unit 34, route acquisition unit 36, storage control unit 38, information update unit 40, current position estimation unit 42, guidance control unit 44, etc. may be implemented using hardware such as a circuit. It may also be configured by Although not shown in FIG. 3, the CPU 14a can also implement various modules necessary for the vehicle 1 to travel. Further, although FIG. 2 mainly shows the CPU 14a that executes the parking assistance process, it may also include a CPU for realizing various modules necessary for the vehicle 1 to run, or it may include an ECU separate from the ECU 14. It's okay.

The situation information acquisition unit 32 acquires surrounding situation information of a surrounding area including the parking target position detected by a detection unit provided in the vehicle 1. The situation information acquisition unit 32 acquires surrounding situation information when parking in a parking lot where it has not been parked in the past, and also acquires surrounding situation information when parking the vehicle 1 repeatedly at the same parking target position. Obtain surrounding situation information. Here, "same parking target position" means, for example, a movement target position set in a home parking lot (garage), a contract parking lot, a predetermined parking lot that is used for car sharing, etc. It is. In other words, the target position is determined in a parking lot where the vehicle 1 is likely to be repeatedly parked, and is determined to keep the vehicle 1 within a predetermined parking area of the parking lot.

The image information acquisition section 32a included in the situation information acquisition section 32 acquires a captured image (surrounding image) showing the surrounding situation of the vehicle 1 including the road surface on which the vehicle 1 exists, which is captured by the imaging section 15, in a learning mode described later. Obtained sequentially when executing `` and automatic guidance mode''. Note that the image information acquisition unit 32a may sequentially acquire images captured by each imaging unit 15 (15a to 15d). In another example, the image information acquisition unit 32a is configured to acquire peripheral situation information about the possible travel direction of the vehicle 1 based on information regarding the possible travel direction of the vehicle 1 (forward direction or backward direction) that can be obtained from the shift sensor 21. , it is also possible to selectively acquire captured images in directions in which travel is possible.

A distance information acquisition unit 32b included in the situation information acquisition unit 32 detects objects (walls, shelves, flower beds, pillars, etc.) around the vehicle 1, including the road surface on which the vehicle 1 is located, which is acquired by the radar 16 and the sonar 17. (structures, other vehicles, pedestrians, etc.). Note that the distance information acquisition unit 32b may sequentially acquire the distance information acquired by each radar 16 (16a to 16d) and sonar 17 (17a to 17h), similarly to the image information acquisition unit 32a. In another example, the distance information acquisition unit 32b acquires distance information to an object existing in the possible travel direction based on information regarding the possible travel direction (forward direction or backward direction) of the vehicle 1 that can be acquired from the shift sensor 21. Distance information on possible travel directions may also be selectively acquired. Note that the situation information acquisition unit 32 may use both image information and distance information as the surrounding situation information, or may acquire either one of them. When both image information and distance information are used, the accuracy and reliability of surrounding situation information can be improved. On the other hand, when one of them is used, the detection section that is not used can be omitted from the system, contributing to cost reduction and processing reduction.

The mode switching unit 34 has a learning mode that is executed when parking the vehicle 1 in a parking lot for the first time (first parking target position), and a learning mode that is executed when parking the vehicle 1 in a parking lot for the first time (first parking target position) A switching process with the automatic guidance mode that is executed when parking the vehicle 1 at the parking target position is executed. The mode switching may be performed by a user (for example, a driver) of the vehicle 1 using the operation input unit 10 or the like. In another example, the mode switching unit 34 determines whether the parking lot (parking target position) is a new parking lot or a parking lot where the parking lot has been parked in the past, based on the captured image acquired by the image information acquisition unit 32a. It may be determined whether the vehicle is in a parking lot and automatically switch between the learning mode and the automatic guidance mode.

For example, when parking is performed in learning mode, that is, when parking the vehicle 1 in a parking lot for the first time, the route acquisition unit 36 acquires the travel route passed by the moving vehicle 1 by the driver's operation. do. Alternatively, the moving route may be sequentially calculated and acquired based on the surrounding image including the parking target position acquired by the image information acquisition unit 32a and the current position of the vehicle 1, as executed by a well-known parking assistance device. good. Alternatively, the peripheral image including the parking target position and the current position of the vehicle 1 may be transmitted to an external system, and the travel route calculated by that system may be received and acquired.

The storage control unit 38 executes a process of storing information acquired when the vehicle 1 parks at the parking target position in a storage device such as the SSD 14f. The situation information storage control unit 38a sequentially stores the surrounding situation information as first situation information that is acquired when the vehicle 1 completes parking at the parking target position. For example, in the learning mode, when the driver manually moves (parks) the vehicle 1 to a parking lot (parking target position), the vehicle 1 reaches the parking target position from the time it reaches the vicinity of the parking lot. (until parking is completed) surrounding image information, distance information to surrounding objects, etc. are sequentially stored as first situation information. Furthermore, as will be described later, even when the first situation information is updated by the second situation information (situation information acquired while the vehicle 1 is moving in automatic guidance mode), the first situation after the update Memorize information.

FIG. 4 is an explanatory diagram exemplarily and schematically showing a three-dimensional space diagram 50 and map data 56 based on first situation information stored at a certain timing. In addition, for the sake of explanation, the three-dimensional space diagram 50 also shows a parking target position T set in a part of the home parking lot P and a movement route R for moving the vehicle 1 to the parking target position T. represented. FIG. 4 shows a home parking lot P surrounded by a wall 52 as an example of a parking lot where the vehicle 1 is parked. When the wall surface 52 is shown on the two-dimensional image captured by the imaging unit 15 mounted on the vehicle 1, a well-known image processing technique is used to extract feature points 54 for identifying the wall surface 52, for example. I can do it. As a technique for extracting the feature points 54, for example, FAST (Features from Accelerated Segment Test) or SIFT (Scale-Invariant Feature Transform) can be used. can do. Further, it is possible to obtain map data 56 including map feature points 54a corresponding to feature points 54 using a well-known image processing technique, such as SLAM (Simultaneous Localization and Mapping). The situation information storage control unit 38a stores at least the map data 56 as first situation information in the SSD 14f or the like.

Therefore, when the same vehicle 1 as the vehicle 1 that acquired the first situation information newly parks at the same parking target position T, the correspondence between the second situation information as the surrounding situation information acquired sequentially and the first situation information By doing so, the current position of the imaging unit 15 (vehicle 1) can be estimated. That is, the feature amount obtained from the feature point 54 extracted from the two-dimensional captured image of the second situation information and the feature amount of each map feature point 54a are associated with the map data 56 stored as the first situation information. By doing so, the position of the imaging unit 15 on the map can be specified. That is, the current position of the vehicle 1 (the current position of the imaging unit 15) in the three-dimensional space can be estimated.

The route storage control unit 38b also stores the movement route of the vehicle 1 until it reaches the parking target position, that is, from the time when the vehicle 1 reaches the vicinity of the parking lot, which is acquired by the route acquisition unit 36, to the parking target position T. The moving route until the vehicle 1 completes its movement (completes parking) is stored. That is, the route storage control unit 38b stores the movement route of the vehicle 1 when the movement of the vehicle 1 is completed without contacting surrounding objects (for example, the wall surface 52, etc.) as a (reference movement route). The storage control unit 38 may store the first situation information and the travel route R as combined information.

Therefore, by comparing the first situation information (stored information) and the second situation information (currently acquired information), the current position of the vehicle 1 can be identified, and the vehicle 1 can be safely moved to the parking target position T. The vehicle 1 can be guided to the parking target position T using the movement route R that has been confirmed to be possible.

By the way, as shown in FIG. 5, consider a case where, for example, a new structure 58 (for example, a shelf) is set in the home parking lot P. In this case, when the same vehicle 1 as the vehicle 1 that has acquired the already stored first situation information newly parks at the same parking target position T in the home parking lot P where the structure 58 is installed, the imaging Assume that the unit 15 acquires a two-dimensional captured image indicating surrounding situation information (second situation information). In this case, the feature points 54 of the wall surface 52 and the feature points 60 of the structure 58 as new information are extracted from the acquired two-dimensional captured image. Therefore, the correspondence between the first situation information (the feature amount of the feature point based on the wall surface 52 included in the stored map data 56) and the second situation information (the feature amount of the feature point based on the wall surface 52 and the structure 58) is As a result, the current position of the vehicle 1 cannot be determined or has low accuracy including errors. As a result, inconveniences may occur, such as the position on the moving route R being unable to be specified, making it impossible to continue parking assistance, and increasing the possibility of contact with the structure 58.

Therefore, if there is a difference between the first situation information and the second situation information, and if a predetermined condition is met, the parking support unit 30 of the present embodiment replaces the first situation information with the second situation information. It includes an information update section 40 for updating. For example, if new information that is not included in the first situation information is included in the second situation information, and the area specified by the new information and the area where the movement route R exists do not interfere, the predetermined condition is satisfied. considered as having been done.

The new information extraction unit 40a includes, for example, at least one of an object boundary line extraction unit 40a1 and a difference extraction unit 40a2. As shown in FIG. 6, the object boundary line extraction unit 40a1 detects a three-dimensional object (for example, a wall surface 52, a feature point 54, etc.) on the road surface included in the first situation information (captured image) and the second situation information (captured image). The object boundary line 62 indicating the boundary between the road surface and the object area recognized as the area in which the object exists is extracted. The object boundary line 62 can be set using a well-known image processing technique, such as edge detection, straight line approximation, machine learning, or the like. Note that the object boundary line 62 may become a non-smooth line depending on the shape and position of the three-dimensional object, the resolution and recognition accuracy of the captured image, noise, and the like. In such a case, a well-known smoothing process (smoothing process) may be applied to the object boundary line 62.

The object boundary line 62 indicates the foot of an object that exists in the home parking lot P (the part where it is installed on the road surface). By reflecting the object boundary line 62 in the map data 56 shown in FIG. 5, the position of the object boundary line 62a and the structure 58 when the structure 58 does not exist are determined as shown in the map data 64 in FIG. A clear difference occurs in the position of the object boundary line 62b when it exists. In other words, new information can be extracted based on the existence of the structure 58.

On the other hand, the difference extraction unit 40a2, for example, compares the feature amount of the feature point of the first situation information with the feature amount of the feature point of the second situation information, and extracts the difference information. For example, feature points 60 of the structure 58 that do not exist in the first situation information shown in FIG. 4, that is, new information can be extracted.

Note that the new information extraction unit 40a may extract new information using either the object boundary line extraction unit 40a1 or the difference extraction unit 40a2, or may extract the new information by using either the object boundary line extraction unit 40a1 or the difference extraction unit 40a2. New information may be extracted using both. By using both the object boundary line extraction section 40a1 and the difference extraction section 40a2, the accuracy of extraction of new information can be improved.

The area determination unit 40b determines whether or not the area specified by the new information, for example, the area increased as the object presence area defined by the object boundary line 62 and the area where the movement route R exists interferes (does they overlap? or not). For example, if the area where the object exists defined by the object boundary line 62 interferes with the area where the moving route R exists, it is determined that the moving route R cannot be used thereafter. In other words, a new travel route R needs to be set. On the other hand, if the area where the object exists defined by the object boundary line 62 and the area where the movement route R exists do not interfere, it is possible to guide the vehicle 1 along the movement route R even if a new structure 58 exists. It is determined that In other words, the moving route R can be used from now on.

Based on the comparison between the first situation information and the second situation information, the update processing unit 40c determines whether new information not included in the first situation information is included in the second situation information and the area specified by the new information. If there is no interference with the area where the moving route R exists, the first situation information is updated with the second situation information including new information. In other words, new information is extracted by the new information extraction unit 40a, and the area in which the object (for example, the structure 58) exists, which is defined by the object boundary line 62 by the area determination unit 40b, does not interfere with the area in which the movement route R exists. If it is determined that the object boundary line 62 is the object boundary line 62, the object information defined by the object boundary line 62 (the feature points 60 of the structure 58, the object boundary line 62b, etc.) is reflected in the first situation information. FIG. 7 exemplarily and schematically shows the map data 66 (first situation information) after being updated using the second situation information (information including the feature points 60 of the structure 58 and the object boundary line 62b). FIG. Therefore, the first situation information (map data 66) to be compared with the second situation information acquired by the situation information acquisition unit 32 in order to guide the vehicle 1 to the parking target position T next time has a structure that is considered to be newly increased. Information regarding the object 58 becomes the information taken into consideration.

The current position estimation unit 42 estimates the current position of the vehicle 1 by comparing the first situation information and the second situation information. The first situation information is surrounding information acquired when the vehicle 1 moves to the parking target position T, and is information that confirms that the vehicle 1 does not come into contact with objects existing in the surroundings. Therefore, when the same vehicle 1 as the vehicle 1 that acquired the first situation information newly parks at the same parking target position T, the second situation information as the surrounding situation information acquired sequentially is associated with the first situation information. By doing so, it is possible to specify the position on the map of the imaging unit 15 that is capturing the captured image corresponding to the second situation information. That is, the current position of the vehicle 1 can be estimated. Furthermore, even when a new structure 58 exists, the first situation information in which information regarding the structure 58 is taken into consideration is generated by the update processing unit 40c. Therefore, even if the vehicle 1 heads toward the parking target position T and newly parks in the current home parking lot P to which the structure 58 has been added, the currently acquired second situation information and the updated first situation By associating the information with the information, the current position of the imaging unit 15 (vehicle 1) can be estimated (specified) with high accuracy.

In this way, the parking support unit 30 of the present embodiment determines that the predetermined condition is satisfied when the new information is extracted and the area corresponding to the travel route R with a track record does not interfere with the area based on the new information. The first situation information is updated using the new information (second situation information). As a result, the first situation information, which is the latest surrounding situation information, can be used in the next parking guidance control, and the current position of the vehicle 1 can be estimated with high accuracy.

The guidance control unit 44 guides the vehicle 1 to the parking target position T according to the estimated current position of the vehicle 1 and the travel route R. The route effectiveness determination unit 44a determines the current effectiveness of the currently stored travel route R, that is, the travel route R that has a track record of guiding the vehicle 1 to the parking target position T. For example, in the area determining unit 40b, if the area where the object exists defined by the object boundary line 62 interferes with the area where the stored moving route R exists, the validity of the moving route R is denied. In other words, it is determined that subsequent use is not possible. In this case, the route validity determination unit 44a may notify the user (driver) using the display device 8 or the audio output device 9 that the currently stored travel route R is not available. In this case, the driver issues a mode switching request to the mode switching unit 34 via the operation input unit 10 or the like to switch to the learning mode. That is, while the vehicle 1 is moved to the parking target position T by the driver's own operation, new first situation information is stored via the situation information storage control section 38a, and the new movement route R is stored as the route. This is done via the storage control unit 38b. In another example, the route acquisition unit 36 executes a new route search (research) and acquires a travel route that avoids the structure 58 and reaches the parking target position T. On the other hand, if the area where the object exists defined by the object boundary line 62 does not interfere with the area where the movement route R exists, even if the structure 58 exists, the movement path R that has been guided and is currently stored is It is determined that it is possible to guide (travel) the vehicle 1 along. In other words, the moving route R can be used from now on. In this case, the guidance control unit 44 uses the display device 8 and the audio output device 9 to inform the user (driver) that the currently stored travel route R is valid and that guidance will be performed according to the travel route R. You may notify.

The guidance processing unit 44b executes processing for guiding the vehicle 1 to the parking target position T based on the current position of the vehicle 1 and the travel route R estimated by the current position estimation unit 42. For example, in order to realize driving support for moving the vehicle 1, control information is provided to the steering system 13, the brake system 18, the drive system 24, etc. For example, the guidance processing unit 44b executes fully automatic control that automatically controls all of the steering system 13, brake system 18, drive system 24, etc., or controls a part of the steering system 13, brake system 18, drive system 24, etc. or perform semi-automatic control. Further, the guidance processing unit 44b provides operation guidance for the steering system 13, brake system 18, drive system 24, etc. to the driver so that the vehicle 1 can move along the travel route R, and provides the driver with driving operation guidance. It is also possible to perform manual control to execute the process. In this case, the guidance processing section 44b may provide operation information to the display device 8 and the audio output device 9. Further, even when performing semi-automatic control, the guidance processing unit 44b can provide information regarding the driver's operation, such as accelerator operation, through the display device 8 and the audio output device 9.

An example of the flow of parking assistance processing by the parking assistance device (parking assistance unit 30) configured as described above will be explained using the flowchart of FIG. 8.

The parking support unit 30 determines whether the current process is a new parking process based on the mode switching state of the mode switching unit 34 (S100). If the parking is not new (No in S100), that is, the automatic guidance mode is selected by the mode switching unit 34, and the travel route used in the past is sent to a parking lot that has been used in the past (for example, home parking lot P). This is a case where parking is executed using R. In this case, the information update unit 40 acquires first situation information, which is past information stored in a storage device such as the SSD 14f (S102). Further, the information update unit 40 acquires a moving route R that has a track record of guiding to the parking target position T corresponding to the first situation information, which is past information stored in a storage device such as the SSD 14f (S104). Subsequently, the situation information acquisition unit 32 uses the imaging unit 15, radar 16, sonar 17, etc. to acquire second situation information (current surrounding situation) (S106).

Then, the new information extraction unit 40a determines whether there is new information based on the comparison between the first situation information acquired from the SSD 14f and the newly acquired second situation information (S108). For example, the information update unit 40 extracts the object boundary line 62 with the object boundary line extraction unit 40a1, and extracts the difference with the difference extraction unit 40a2, thereby generating new information (for example, regarding the structure 58, etc.). information).

In S108, if it is determined that there is new information (Yes in S108), the area determination unit 40b determines whether the area specified by the new information and the movement route R that has a track record of guiding to the parking target position T acquired from the SSD 14f is It is determined whether or not there is interference with the existing area (S110). When the area determination unit 40b determines that there is no interference between the area based on the new information and the area based on the movement route R (No in S110), the update processing unit 40c uses the second situation information including the new information to 1 status information is updated (S112). That is, new information (for example, information such as the object boundary line 62 including the structure 58) is reflected in the first situation information to become new first situation information.

The current position estimation unit 42 estimates the current position of the vehicle 1 based on the currently acquired second situation information and the updated first situation information (S114), and the guidance control unit 44 starts guidance control. (S116). That is, if the area determining unit 40b determines that there is no interference between the area based on the new information and the area based on the moving route R, the route validity determining unit 44a determines that the currently referenced moving route R is valid. The guidance processing unit 44b sequentially guides the vehicle 1 whose current position is recognized based on the movement route R. If the vehicle 1 has not reached the parking target position T (No in S118), the guidance control unit 44 returns to S114 and again determines the position of the vehicle 1 based on the comparison between the first situation information and the second situation information. The current position is estimated and the guidance control of the vehicle 1 following the moving route R is continuously performed. On the other hand, when the vehicle 1 reaches the parking target position T (Yes in S118), the guidance control unit 44 executes a guidance completion process (S120). For example, the display device 8 or the audio output device 9 is used to notify the driver that the movement to the parking target position T is completed, and the storage control unit 38 updates the latest first situation reflecting the new information. The information is stored in a storage device such as the SSD 14f so that it can be used in the next parking assistance control, and the series of parking assistance processing is temporarily terminated.

Note that if it is determined in S110 that there is interference between the area based on the new information and the area based on the travel route R (Yes in S110), the guidance control unit 44 determines that the stored travel route R is based on the current parking lot. (For example, home parking lot P) is determined to be unusable, and the process moves to S120 to notify that the stored travel route R is unusable as guidance completion processing. For example, a message may be output to the driver to direct the driver to drive towards the parking target position T while checking the surroundings, or a message may be output by the route acquisition unit 36 to guide the driver to search for a new travel route R. The series of parking support processing is temporarily ended. In this case, the storage control unit 38 may perform a process of temporarily discarding the first situation information and the movement route R stored in the SSD 14f or the like.

Further, in S108, if new information is not extracted by the information update unit 40, that is, if the stored first situation information and the currently acquired second situation information correspond, the information is updated. The unit 40 maintains the first status information as non-updated. That is, the stored first situation information is used as is for estimating the current position of the vehicle 1. Therefore, proceeding to S114, the current position estimating unit 42 estimates the current position of the vehicle 1 based on the first situation information acquired from the SSD 14f and the second situation information currently acquired, and performs the subsequent processing. Continue to do so.

Further, in S100, if it is determined that the current parking is a new parking process (Yes in S100), the situation information acquisition unit 32 uses the imaging unit 15, radar 16, etc. when the vehicle 1 heads toward the parking target position T. , the new first situation information obtained using the sonar 17 or the like (S122), and the situation information storage control unit 38a sequentially stores it in a storage device such as the SSD 14f. Further, the travel route of the vehicle 1 at that time is sequentially acquired (S124), and the route storage control unit 38b stores it in a storage device such as the SSD 14f, and once ends the series of parking support processing. That is, the next time the vehicle is parked at the same parking target position T in the same parking lot in the automatic guidance mode, the first situation information and travel route R stored this time will be used.

As described above, according to the parking assistance device of the present embodiment, a change (difference) is confirmed in the surrounding situation information, and an area specified by the new information interferes with an area where the stored travel route R exists. If not, it is assumed that the predetermined condition is satisfied, and the first situation information is updated at that timing. That is, if the travel route R is still usable from now on, the first situation information that can be used to estimate the current position of the vehicle 1 is updated at that timing. As a result, even when performing parking assistance control next time, it is possible to correspond between the stored first situation information and the currently acquired second situation information, and the current position of the vehicle 1 can be accurately estimated. , can perform high-precision parking assistance.

In addition, although the above-mentioned embodiment shows the case where the car is parked in the home parking lot P, the present embodiment can be applied to cases where the car is repeatedly parked in the same position. This embodiment can also be applied when parking in a parking lot used for, etc., and similar effects can be obtained.

The parking assistance program for parking assistance processing executed by the parking assistance unit 30 (CPU 14a) of this embodiment is a file in an installable format or an executable format, and is stored on a CD-ROM, a flexible disk (FD), or a CD-ROM. The information may be provided by being recorded on a computer-readable recording medium such as R, DVD (Digital Versatile Disk), or the like.

Furthermore, the parking assistance program may be stored on a computer connected to a network such as the Internet, and provided by being downloaded via the network. Further, the object detection program executed in this embodiment may be provided or distributed via a network such as the Internet.

Although the embodiments and modifications of the present invention have been described, these embodiments and modifications are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and their modifications are included within the scope and gist of the invention, as well as within the scope of the invention described in the claims and its equivalents.

1...Vehicle, 14...ECU, 14a...CPU, 14f...SSD, 15...Imaging unit, 16...Radar, 17...Sonar, 30...Parking support unit, 32...Situation information acquisition unit, 32a...Image information acquisition unit, 32b ...Distance information acquisition unit, 34...Mode switching unit, 36...Route acquisition unit, 38...Storage control unit, 38a...Situation information storage control unit, 38b...Route storage control unit, 40...Information update unit, 40a...New information extraction Part, 40a1... Object boundary line extraction unit, 40a2... Difference extraction unit, 40b... Area determination unit, 40c... Update processing unit, 42... Current position estimation unit, 44... Guidance control unit, 44a... Route validity determination unit, 44b ...Guidance processing unit, 62, 62a, 62b...Object boundary line, R...Movement route, T...Parking target position.

Claims (3)

a situation information acquisition unit that acquires surrounding situation information of a surrounding area including the parking target position detected by a detection unit provided in the vehicle when repeatedly parking a vehicle at the same parking target position;
a storage control unit that stores first situation information indicating the surrounding situation information acquired when the vehicle completes parking at the parking target position and a travel route until reaching the parking target position; and,
The first situation information is compared with second situation information indicating the surrounding situation information acquired when the vehicle newly parks at the parking target position, and new information not included in the first situation information is detected. Based on the comparison result, it is determined whether the area specified by the second situation information and the area specified by the new information and the area where the movement route exists interferes , and the area specified by the new information is determined based on the comparison result. an information updating unit that updates the first situation information stored by the storage control unit with the second situation information including the new information when there is no interference between the area where the movement route exists and the area where the movement route exists;
a current position estimation unit that estimates the current position of the vehicle by comparing the first situation information and the second situation information;
a guidance control unit that guides the vehicle to the parking target position according to the estimated current position and the travel route;
A parking assistance device equipped with The information updating unit extracts an object boundary line that can be considered as a boundary line between a road surface and an object included in the first situation information and the second situation information, and updates the first situation information and the second situation information. A parking assist device according to claim 1 for comparison. The parking system according to claim 1 or 2, wherein the information update unit extracts difference information between the first situation information and the second situation information, and updates the first situation information using the difference information. Support equipment.

Images