JPH10244890A - Automatic parking device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect a parking block and guide a vehicle by calculating the parking path to a parking point based on the processed results of the rear, right, and left images of the vehicle photographed by three image pickup means. SOLUTION: An image processing device 4 processes the images photographed by cameras 1-3, detects the white lines indicating a parking block on the road surface, and detects the lower sections of the side faces of a parking vehicle at a parking lot partitioned with the parking block by ropes stretched on an unpaved surface. A surrounding environment recognizing device 5 recognizes the parking block and the vehicle surrounding environment based on the image processed results by the image processing device 4. A parking control device 6 controls a steering control device 7, a braking force control device 8, a driving force control device 9 automatically changing the driving force, and an automatic transmission 10 automatically switching a lever to advance D, reverse R, neutral N, or parking P to calculate the path to a parking point based on the parking block and vehicle surrounding environment recognized by the surrounding environment recognizing device 5 and guide the vehicle along the parking path.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic parking system for detecting a parking spot and guiding a vehicle to the parking spot.

[0002]

2. Description of the Related Art As shown in FIG. 19, the surrounding environment behind a vehicle is imaged by a camera, the front corner of a parked vehicle is detected by image processing, and a parking point is located between the front corners of adjacent parked vehicles. There is known an automatic parking device that guides a vehicle by setting (for example, see Japanese Patent Application Laid-Open No. 5-143895).

[0003]

By the way, as shown in FIG. 20, there is also a narrow parking lot where the turning amount of the vehicle cannot be increased. When performing automatic parking using a conventional automatic parking device in such a parking lot, the vehicle cannot be moved to an initial position where the front corners of the parked vehicles adjacent to the target parking section can be imaged, that is, an automatic parking start point. There is a problem that automatic parking is not possible. Further, as shown in FIG. 21, if the vehicle stops almost perpendicularly to the parking direction of the parking section and is set as the automatic parking start point, the front corners of the parked vehicles on both sides may not be accurately detected, and the automatic parking may be performed. There is a problem that can not be.

It is an object of the present invention to provide an automatic parking device that accurately detects a parking section, calculates a parking route, and guides a vehicle.

[0005]

[Means for Solving the Problems]

(1) The invention according to claim 1 is a steering control means for automatically steering a steered wheel, a braking force control means for automatically generating a braking force, a driving force control means for automatically changing a driving force,
Automatic shifting means for automatically switching forward, backward, neutral and parking; three imaging means for imaging the rear, left and right sides of the vehicle, respectively; and image processing means for processing an image captured by the imaging means; A path calculating means for calculating a parking route to a parking point based on an image processing result by the image processing means; a steering control means, a braking force control means, a driving force control means, and an automatic transmission means; Parking control means for automatically moving and parking the vehicle. The rear, left and right sides of the vehicle are imaged by the three imaging means, a parking route to a parking point is calculated based on the processing results of the captured images, and the vehicle is automatically moved along the parking route. Move and park. (2) In the automatic parking device according to a second aspect, the left-side imaging unit and the right-side imaging unit are arranged near the driver's eyes in the front-rear direction of the vehicle. (3) In the automatic parking device according to the third aspect, the image processing means detects a white line of the parking lot or a side surface of the parked vehicle on both sides to recognize the parking section. (4) In the automatic parking device according to the fourth aspect, a path between the intersections of the extension lines of the left and right parking sections and the side of the vehicle is set as a virtual entrance of the parking section, and a path passing through the virtual entrance is calculated by the path calculating means. It is like that. (5) In the automatic parking device according to a fifth aspect, the path calculating means calculates a parking path composed of an arc having a minimum turning radius and a straight line. (6) In the automatic parking device according to the sixth aspect, the path calculation means advances in a direction away from the parking section along the arc of the minimum turning radius, and then rides on the center line of the parking section along the arc of the minimum turning radius. The parking route is calculated. (7) In the automatic parking device according to claim 7, the distance to the obstacle on the opposite side to the parking section is detected by the image processing means by image processing, and the distance to the obstacle on the opposite side to the parking section is detected by the path calculation means. The calculated parking route is taken into account.

[0006]

【The invention's effect】

(1) According to the first aspect of the present invention, the rear, left, and right sides of the vehicle are imaged by the three imaging units, and the parking route to the parking point is calculated based on the processing results of the captured images. The vehicle is automatically moved along the parking route to park the vehicle, so that compared to the conventional case where one camera captures an image of the surrounding environment of the vehicle, how the vehicle is automatically parked in the initial position Even if the vehicle is stopped in an appropriate direction, the front corners of the vehicles on both sides of the target parking section can be accurately detected, and automatic parking can be performed even in a narrow parking lot. (2) According to the second aspect of the present invention, since the left-side imaging unit and the right-side imaging unit are disposed near the driver's eyes in the front-rear direction of the vehicle, the driver's own line of sight is between the parking sections. It is possible to easily set the correct initial position of the automatic parking simply by stopping the vehicle. (3) According to the third aspect of the invention, the parking section is recognized by detecting the white line of the parking lot or the side of the parked vehicle on both sides by image processing, so that the parking section can be accurately detected. (4) According to the invention of claim 4, the virtual entrance of the parking section is set between the intersection between the extension line of the left and right parking sections and the side of the vehicle, and the route passing through the virtual entrance is calculated. In order to reduce the conversion error from the screen coordinate system to the road coordinate system, even if a lens with a small focal length and a small focal length is used for the image capturing means, the entrance to the parking section can be specified, and the image with little distortion can be obtained. The screen can be processed to calculate an accurate parking route. (5) According to the invention of claim 5, since the parking path composed of the arc and the straight line having the minimum turning radius is calculated, it is possible to move to the parking point along the shortest path and even in a narrow parking lot. Automatic parking becomes possible. (6) According to the sixth aspect of the present invention, a parking path that proceeds in a direction away from the parking section along the arc of the minimum turning radius and then rides on the center line of the parking section along the arc of the minimum turning radius is calculated. As a result, an effect similar to that of the fifth aspect is obtained. (7) According to the seventh aspect of the present invention, the distance to the obstacle on the opposite side to the parking section is detected by image processing, and the parking route is calculated in consideration of the distance to the obstacle.
Automatic parking is possible even in a small parking lot.

[0007]

FIG. 1 shows a configuration of an embodiment. The rear camera 1 captures the surrounding environment behind the vehicle, the left camera 2 captures the surrounding environment on the left side of the vehicle, and the right camera 3 captures the surrounding environment on the right side of the vehicle. Camera 1
3 uses a wide-angle black-and-white camera with little distortion around the screen.
These cameras 1 to 3 are installed at a place with a large ground height in order to secure a wide field of view as possible at a fixed angle of view. In particular, the rear camera 1 is installed at the center of the rear end of the vehicle in the vehicle width direction so as to be able to cope with either the left or right parking lot. The left and right side cameras 2 and 3 are installed at positions near the driver's eyes (eye points) in the vehicle front-rear direction as shown in FIG. 2 so that the vehicle can be easily stopped at an optimum automatic parking start point. .

FIG. 3 shows an installation example of a camera when an automatic parking device is mounted on a passenger car type vehicle called a three-box car. The rear camera 1 is buried inside the rear end of the trunk lid. Further, the left camera 2 is embedded inside the roof at a position close to the eyes of the occupant in the vehicle longitudinal direction on the left side of the vehicle. Further, the right camera 3 is embedded inside the roof at a position close to the eyes of the occupant in the vehicle front-rear direction on the right side of the vehicle.

FIG. 4 shows an installation example of a camera when an automatic parking device is mounted on a vehicle called a one-box car.
The rear camera 1 is embedded near the hinge of the rear hatch at the rear end of the vehicle. Further, the left camera 2 is embedded inside the roof at a position close to the eyes of the occupant in the vehicle longitudinal direction on the left side of the vehicle. Further, the right camera 3 is embedded inside the roof at a position close to the eyes of the occupant in the vehicle front-rear direction on the right side of the vehicle.

The image processing device 4 processes images taken by the cameras 1 to 3 and detects a white line representing a parking section drawn on a road surface as shown in FIG. That is, the captured image is converted into a differential image using a sobel filter or the like, a set of substantially parallel edge components forming a white line is detected, and a white line is present when the interval between the one set of edge components is brighter than another road surface. Recognize.

The image processing device 4 processes images captured by the cameras 1 to 3 and parks the vehicle in a parking lot where a parking section is partitioned by stretching a rope on a non-paved surface as shown in FIG. Detect lower side of vehicle. That is, the captured image is converted into a differential image using a Sobel filter or the like, and the boundary between the shadowed portion of the parked vehicle and the road surface is detected as an edge component, and the darker portion than the road surface is recognized as the shadow of the parked vehicle. Then, the lower side of the parked vehicle is detected.

The surrounding environment recognizing device 5 recognizes a parking space or a vehicle surrounding environment based on the image processing result of the image processing device 4. The parking control device 6 includes a microcomputer and its peripheral components, calculates a route to a parking point based on the parking section and the vehicle surrounding environment recognized by the surrounding environment recognition device 5, and guides the vehicle along the parking route. The steering control device 7, the braking force control device 8, the driving force control device 9, and the automatic transmission 10 are controlled as needed. The steering control device 7 is a device that automatically steers a steered wheel, and the braking force control device 8 is a device that automatically generates a braking force. The driving force control device 9 is a device that automatically changes the driving force, and the automatic transmission 10 is a device that automatically switches between forward D, reverse R, neutral N, and parking P.

The left and right parking lot selection switch 11 is a switch for selecting whether to park in the parking lot on the right side of the vehicle or in the parking lot on the left side of the vehicle. The automatic parking start switch 12 starts automatic parking. Switch. The display device 13 displays images captured by the cameras 1 to 3.

FIG. 7 is a flowchart showing the automatic parking control executed by the parking control device 6. The operation of the embodiment will be described with reference to this flowchart. The occupant moves through the parking lot while searching for a parking place, and when he finds a vacant parking space, as shown in FIG. 8, the occupant automatically moves his or her eyes between the parked vehicles on both sides or between empty parking spaces. At the initial position of parking (automatic parking start point), the vehicle stops in a direction substantially perpendicular to the parking direction of the parking section. Then, the shift lever is set to the parking P or neutral N position, and the brake pedal is released to make the steering neutral. When the preparation for the above automatic parking is completed, the occupant selects the right or left parking lot by the left and right parking selection switch 11, and then operates the automatic parking start switch 12 to start the automatic parking operation.

When the automatic parking start switch 12 is turned on, the parking control device 6 starts automatic parking control after step 2. In step 3, left and right parking lot selection switch 1
The parking lot selected by 1 is read.

Here, the method of starting the automatic parking operation is not limited to the above-described method using the dedicated switch. For example, a shift position dedicated to automatic parking is provided, and when the shift position is set, the automatic parking operation is started. You may. Further, the automatic parking operation may be automatically started when a predetermined operation is performed, such as setting the shift lever to parking P or neutral N and releasing the brake pedal to return the steering to neutral. In this case, the automatic parking control is canceled unless parking conditions such as left and right parking position selection are set.

Next, at step 4, the surrounding environment recognition device 5 calculates the width W of the parking section, the relative position L between the parking section and the vehicle, and the relative angle θ shown in FIG. Since the side cameras 2 and 3 are arranged at positions near the driver's eyes in the vehicle front-rear direction, images shown in FIG. 10 are captured by the side cameras 2 and 3 at the automatic parking start point. In this image, a white line indicating the side surface or the parking section of the parked vehicle on both sides is detected, and a straight line formula on the screen is obtained. In order to obtain the straight line equation of the white line, as shown in FIG. 11, a small area for detecting the white line is set on the screen, and the white line representing the parking section is detected in the area as described above. The start point and the end point of the inner (parking section side) edge of the two edges of the left and right white lines are specified, and the start point coordinates (x1, y1) on the screen,
(X3, y3) and end point coordinates (x2, y2), (x4, y
Find 4).

FIG. 12 is a diagram for explaining the relationship between the coordinate system x, y on the screen of the display device 13 and the actual road coordinate system X, Y, Z. Assuming that the ground clearance of the lenses of the cameras 1 to 3 is H, the focal length of the lenses of the cameras 1 to 3 is F, and the pitch angle between the optical axes of the lenses of the cameras 1 to 3 is α,

X = −FX / {− (Y−H) sinα + Zcosα}, y = −F {(Y−H) cosα + Zsinα} / ｛−
(Y−H) sinα + Zcosα｝ Since the white lines captured by the cameras 1 to 3 are on the road surface, Y =
The start point (x1, y1), (x3, y3) and end point (x2, y2), (x4, y4) of the white line edge shown in FIG. Can be converted to

From the start point and the end point converted to the road coordinate system, a straight line equation of a white line edge representing two parking sections can be calculated.
Here, since Y = 0, the linear equation of the white line edge is a linear equation of X and Z. The width W of the parking section, the relative position L between the parking section and the vehicle, and the relative angle θ can be obtained from the linear formula of the white line edge in the road coordinate system. That is, the width W is Z = two straight lines representing the parking section.
It is calculated as the difference between the values of X at zero. The relative position L is obtained by calculating the center line of the parking section from the average of the X values of two straight lines representing the parking section at Z = 0, and determining the center line and the position of the rear axle (the positional relationship between the camera 2 and 3). Is calculated as the difference from the known value. Further, the relative angle θ is obtained as an average value of the inclinations of the two straight lines representing the parking sections.

The white line of the parking lot is detected to detect the width W of the parking section, the relative position L between the parking section and the vehicle, and the relative angle θ.
However, the width W of the parking section, the relative position L between the parking section and the vehicle, and the relative angle θ may be calculated by detecting the side of the parked vehicle from the shadow of the parked vehicle on both sides.

In step 5, it is determined whether parking is possible based on the width W of the parking space. That is, as shown in FIG. 13, if there is no space S enough for the occupant to open the door and get on and off after parking, the process proceeds to step 6, and the display and the sound warn that there is not enough space in this parking lot to issue an automatic warning. The parking control ends.

If the parking lot has a sufficient width for the occupant to get on and off, the process proceeds to step 7, and a distance D that can move to the opposite side of the parking lot during automatic parking control is obtained as shown in FIG.
Now, it is assumed that an image as shown in FIG. 15 is captured by the side camera on the opposite side to the parking lot. Search from the bottom of the screen to detect the lowest obstacle on the screen, such as a parked vehicle or wall. In the case of a parked vehicle, the y-coordinate on the screen in front of the vehicle is obtained from the shadow under the parked vehicle by image processing. For the wall, the boundary between the road surface and the wall is detected by image processing, and the y coordinate of the wall on the screen is obtained. On an actual road surface, near objects appear on the screen below, and objects far away appear on the screen. Therefore, based on the y-coordinate of the lower end of the obstacle on the screen, the distance to the actual obstacle on the road surface can be obtained by the above equation (1).
Assuming that the Y coordinate of the road coordinate system is the road surface (Y = 0) in Expression 1, Expression 1 is a relational expression between y in the screen coordinate system and Z in the road coordinate system. After the distance to the nearest obstacle on the road is calculated, an appropriate margin is subtracted from the distance to set a distance D that can be moved to the opposite side of the parking.

In step 8, based on the calculated width W of the parking section, the relative position L and the relative angle θ between the parking section and the vehicle, and the moving distance D to the opposite side of the parking lot, the traveling route to the parking point is determined. Calculate. First, as shown in FIG. 14, the minimum turning radius that can contact the vehicle center line at the initial position of automatic parking (automatic parking start point) and move to the opposite side of the parking lot by the distance D that can move to the opposite side of the parking lot. Arc A of
1 is calculated. The arc A1 is an arc that can be translated in a direction along the vehicle center line at the initial position of the automatic parking. Next, as shown in FIG. 16, the width W of the parking section, the relative position L between the parking section and the vehicle, and the relative angle θ indicate the route that contacts the center line of the parking lot and allows the vehicle to pass through the entrance of the parking lot. An arc A2 having a minimum turning radius is calculated. The arc A2 is an arc that can move in parallel in a direction along the center line of the parking lot.

Here, if the focal lengths of the lenses of the side cameras 2 and 3 are shortened, a wider range can be imaged. However, the peripheral portion of the screen is distorted and the conversion error from the screen coordinate system to the road coordinate system increases. . Therefore, as the lenses of the side cameras 2 and 3, lenses having a focal length with a small screen distortion are used. Since a wide-angle image cannot be obtained with a lens having a small focal length, the entrance of the parking section is hardly seen as shown in FIG. Therefore, in this embodiment, as shown in FIG. 16, the intersections of the extension of the white line or the extension of the side of the parking vehicle adjacent to the vehicle and the side of the vehicle on the parking section side of the vehicle are defined as both ends of the entrance of the parking lot.

Next, while moving the arc A1 along the vehicle center line at the initial position of automatic parking (automatic parking start point), the arc A2 is moved along the center line of the parking section, and the arcs A1 and A2 are moved. The route that touches is determined, and the route is set as a parking route. The arc A1 is a path that moves with a minimum turning radius after going straight (retreating) from the automatic parking start point,
It touches the vehicle center line at the automatic parking start point. On the other hand, the arc A2 is a path that can enter the parking section with the minimum turning radius, contacts the center line of the parking section, and passes through the entrance of the parking section.
As shown in FIG. 17, if the route is determined so that the arcs A1 and A2 are in contact with each other, a parking route is constituted only by a straight line and a circle having a minimum turning radius. In the example shown in FIG. 17, parking can be performed in the order of straight-ahead retreat → right-hand steered forward → left-hand steed retreat → straight-forward retreat.

When there are a plurality of points where the arcs A1 and A2 are in contact with each other and a parking route cannot be determined by the above conditions alone, for example, the shortest route may be used as the parking route or the vehicle may enter the parking lot straight. The route may be a parking route.

In step 9, it is confirmed whether or not the parking route has been calculated. If the parking route cannot be calculated, the process proceeds to step 10. In step 10, a warning is issued by a display or voice to move the vehicle toward or away from the parking lot or to select another parking lot, and the automatic parking control ends.

Step 11 if the parking route is calculated
Then, the steering control device 7, the braking force control device 8, the driving force control device 9 and the automatic transmission 10 are controlled to cause the vehicle to travel along the parking route as shown in FIG. In step 12, it is confirmed whether or not the parking has been completed at a predetermined parking point by the brake operation or the operation of the end switch by the occupant, and the traveling control is continued until the parking is completed. It should be noted that the automatic parking end may be recognized by recognizing the completion of parking by operating the shift lever to the P position, operating the parking brake, and processing the image of the rear camera 1 and terminating the automatic parking control.

In the configuration of the above embodiment, the steering control device 7 controls the steering control device, the braking force control device 8 controls the braking force control device, the driving force control device 9 controls the driving force control device,
The automatic transmission 10 constitutes automatic transmission means, the cameras 1 to 3 constitute imaging means, the image processing device 4 constitutes image processing means, the surrounding environment recognition device 5 and the parking control device 6 constitute route calculation means and parking control means, respectively. I do.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an embodiment.

FIG. 2 is a diagram showing an arrangement of a rear camera and a side camera.

FIG. 3 is a diagram showing an example of installation of a camera when an automatic parking device is mounted on a passenger car type vehicle called a three-box car.

FIG. 4 is a diagram illustrating an example of installation of a camera when an automatic parking device is mounted on a vehicle called a one-box car.

FIG. 5 is an explanatory diagram of a method of detecting a white line representing a parking section from a captured image of a parking lot.

FIG. 6 is an explanatory diagram of a method of detecting a side surface of a parked vehicle on both sides recognized as a parking section from a captured image of a parking lot.

FIG. 7 is a flowchart of automatic parking control.

FIG. 8 is a diagram showing a state where the vehicle is stopped at an automatic parking start point.

FIG. 9 is a diagram showing a width W of the parking section, a relative position L between the parking section and the vehicle, and a relative angle θ.

FIG. 10 is a diagram showing an image captured by a side camera at an automatic parking start point.

FIG. 11 is a diagram illustrating a method for obtaining two points on a white line representing a parking ward angle from an image captured by a side camera at an automatic parking start point.

FIG. 12 is a diagram showing a relationship between a screen coordinate system and a road coordinate system.

FIG. 13 is a diagram illustrating a boarding / alighting margin for determining whether parking is possible.

FIG. 14 is a diagram illustrating a method of calculating a distance that can be moved to a side opposite to a parking lot during automatic parking control.

FIG. 15 is a diagram illustrating a method of calculating a distance to an obstacle on the opposite side of a parking lot.

FIG. 16 is a diagram illustrating a method of determining a virtual entrance of a parking space.

FIG. 17 is a diagram illustrating a method of determining a parking route.

FIG. 18 is a diagram showing a parking route.

FIG. 19 is a diagram showing an automatic parking method of a conventional automatic parking device.

FIG. 20 is a diagram illustrating a problem of a conventional automatic parking device in a narrow parking lot where a turning amount of the vehicle cannot be increased.

FIG. 21 is a diagram illustrating a problem of the conventional automatic parking device when the vehicle is stopped substantially perpendicular to the parking direction.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Rear camera 2 Left camera 3 Right camera 4 Image processing device 5 Ambient environment recognition device 6 Parking control device 7 Steering control device 8 Braking force control device 9 Driving force control device 10 Automatic transmission device 11 Left and right parking lot selection switch 12 Automatic Parking start switch 13 Display device

Claims (7)

[Claims] 1. A steering control means for automatically steering a steered wheel; a braking force control means for automatically generating a braking force; a driving force control means for automatically changing a driving force; Automatic shifting means for automatically switching between parking and parking; 3 for imaging the rear, left and right sides of the vehicle, respectively
An image processing means for processing a captured image by the imaging means; a path calculation means for calculating a parking route to a parking point based on an image processing result by the image processing means; and the steering control means; An automatic parking device, comprising: a parking control unit that controls the braking force control unit, the driving force control unit, and the automatic transmission unit to automatically move and park the vehicle along the parking path. . 2. The automatic parking device according to claim 1, wherein the left-side imaging unit and the right-side imaging unit are arranged near the driver's eyes in the front-rear direction of the vehicle. Automatic parking device. 3. The automatic parking device according to claim 1, wherein the image processing unit recognizes a parking section by detecting a white line of a parking lot or a side of a parked vehicle adjacent to the parking lot. Automatic parking equipment. 4. The automatic parking device according to claim 3, wherein the path calculation unit sets a virtual entrance of the parking section between an intersection between an extension of the left and right parking sections and a side of the vehicle, and the virtual entrance. An automatic parking device characterized by calculating a route passing through the vehicle. 5. The automatic parking apparatus according to claim 4, wherein the path calculation means calculates a parking path including an arc having a minimum turning radius and a straight line. 6. The automatic parking device according to claim 5, wherein the path calculation means proceeds in a direction away from the parking section along an arc having a minimum radius of rotation, and then proceeds along the arc having a minimum radius of rotation. An automatic parking device for calculating a parking route on a center line of a parking section. 7. The automatic parking device according to claim 6, wherein the image processing means detects a distance to an obstacle on the opposite side to the parking section by image processing, and the path calculating means is opposite to the parking section. An automatic parking device, which calculates a parking route in consideration of a distance to a side obstacle.