KR20090010669A - Parking assistance system - Google Patents

Abstract

As a parking assistant system, detecting a pedestrian the park route and alarming through the user interface by an electronic control unit. A parking assistant system comprises a user interface(204) informing the information including the peripheral image of the vehicles to the operator of the vehicles; a camera taking a picture of the peripheral image of the vehicles; a stripe generator generating the stripe light to the neighboring of the vehicles; and an electronic control unit(ECU) alarmed through the user interface.

Description

Parking Assistance System

The present invention relates to a parking assistance system of a vehicle. More particularly, the present invention relates to a parking assistance system for recognizing a parking available position in parking a vehicle and of alerting a pedestrian on a parking path during parking to a recognized parking position.

In general, women and novice drivers have considerable difficulties when parking in parking lots and in some cases cause contact accidents. Thus, parking problems can put pressure on many driving inexperienced people and even avoid driving.

One way to solve this problem is to install a camera at the rear and transmit the rear image of the vehicle to the monitor installed in the driver's seat during the parking process, helping the driver to be aware of the parking process in progress. A method to prevent was introduced. However, this has the advantage of being able to grasp more of the situation behind the rearview mirror, but the basic principle is similar to the rearview mirror in that the driver grasps all the situation.

As another method for solving the above problems, a system has been developed that can recognize three-dimensional information of a parked vehicle and recognize an empty space as a parking position by using binocular stereo or motion stereo. However, the operation is guaranteed in a bright place where the image quality is secured, but there is a problem in that it is not guaranteed accurate and reliable operation in a dark place such as an underground parking lot.

In addition, the above two systems do not automatically recognize when a person suddenly appears on the parking path while parking, and there is a possibility of causing an accident during parking without sufficient attention.

In order to solve the above-mentioned problem, the present invention, in parking the vehicle, as well as to recognize the parking available position in the dark parking lot, as well as to alert when the pedestrian appears on the parking path during the parking to the recognized parking position its main purpose There is this.

In order to achieve the above object, the present invention provides a parking assistance system, a user interface for informing a driver of a vehicle of information including a surrounding image of a vehicle, a camera for photographing a surrounding image of the vehicle, and a surrounding of the vehicle. A stripe generator that generates stripe light and a stripe generator are turned on or off, and a signal is transmitted to the camera to take a picture of the surrounding image, and then the parking position of the vehicle is analyzed by analyzing the surrounding image obtained by the camera. It provides a parking assistance system characterized in that it comprises an electronic control unit (ECU) to find and alert through the user interface when a pedestrian is detected on the parking path while the vehicle is parked to the parking position. do.

In addition, in order to achieve the above object, the present invention, in the parking assistance method, (a) obtaining the image when the stripe generator is mounted on the vehicle and the image when the stripe generator is turned off, (b) the Obtaining a stripe image from a difference between an image when the stripe generator is turned on and an image when the stripe generator is turned off, and (c) an error due to an angle between the plane formed by the stripe light emitted from the stripe generator and the ground Correcting to obtain distance data from the stripe image, (d) recognizing the parking position from the distance data, (e) recognizing that parking is started at the parking position, and (f) stripping generator To turn on the image and turn off the stripe generator (G) acquiring a stripe image from a difference between an image of turning on the stripe generator and an image of turning off the stripe generator, and (h) an angle between the plane formed by the stripe light emitted from the stripe generator and the ground Obtaining distance data from the stripe image by correcting an error; (i) determining whether there is a distance data above a threshold value; (j) if there is a distance data above a threshold value as a result of the determination of step (i); proceeding to (l), (k) proceeding to step (f) if there is no distance data above the threshold as a result of the determination of step (i), and (l) detecting the pedestrian by the user interface mounted on the vehicle It provides a parking assistance method comprising the step of notifying.

In order to achieve the above object, the present invention provides a method for setting a parking position of a vehicle, the method comprising: (a) acquiring an image when the stripe generator mounted on the vehicle is turned on and an image when the stripe generator is turned off; (b) obtaining a stripe image from a difference between an image when the stripe generator is turned on and an image when the stripe generator is turned off; (c) an angle at which a plane formed by the stripe light emitted from the stripe generator forms the ground; And obtaining distance data from the stripe image by correcting an error due to the error, and (d) recognizing the parking position from the distance data.

In addition, in order to achieve the above object, the present invention, in the pedestrian warning method of vehicle parking, (a) acquiring the image when the stripe generator mounted on the vehicle and the image when the stripe generator is turned off, (b) obtaining a stripe image from a difference between an image when the stripe generator is turned on and an image when the stripe generator is turned off; (c) an angle at which a plane formed by the stripe light emitted from the stripe generator forms the ground; Obtaining distance data from the stripe image by correcting an error due to the step (d) determining whether there is a distance data greater than or equal to a threshold value, and (e) determining whether the distance data greater than or equal to the threshold value is determined as a result of the step (d). If present, proceeding to step (g), (f) determination of step (d) And if there is no distance data above the threshold, proceeding to step (a) and (g) notifying that the pedestrian is detected by the user interface mounted on the vehicle. Provide a method.

According to the present invention, when parking a vehicle, it is possible to recognize a parking available position even in a dark parking lot, thereby enhancing the driver's parking convenience and alerting when a pedestrian appears on the parking path while parking to the recognized parking position. It is effective to increase.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. First of all, in adding reference numerals to the components of each drawing, it should be noted that the same reference numerals are used as much as possible even if displayed on different drawings. In addition, in describing the present invention, when it is determined that the detailed description of the related well-known configuration or function is obvious to those skilled in the art or may obscure the gist of the present invention, the detailed description thereof will be omitted.

1 is a block diagram illustrating a semi-automatic parking assistance system to which a parking position setting and a pedestrian warning system during parking of the vehicle of the present invention are applied.

The semi-automatic parking assistance system of FIG. 1 comprises a parking lot recognition system 102, a user interface 104, a parking assist controller 106, a vehicle attitude sensor 108, and an active steering system 110.

As shown in FIG. 1, when the driver indicates the intention to park through the user interface 104, the parking lot recognition system 102 recognizes the location information of the available parking place and transmits it to the parking assistance controller 106. The parking assistance controller 106 controls the active steering system 110 to move to the transmitted parking position information to park while actively steering. At this time, the driver is responsible for the safety of the front and side situations that may occur during braking control and parking. Meanwhile, the parking assist controller 106 obtains driving information necessary for steering control from the vehicle attitude sensor 108 including a yaw rate sensor (not shown), a steering angle sensor (not shown), a vehicle speed sensor (not shown), and the like.

2 is a view showing a parking assistance system according to a preferred embodiment of the present invention.

The parking assistance system according to the preferred embodiment of the present invention includes an ECU 202 having an image processor and a controller, a user interface 204, a stripe generator 206, and a camera 208.

As shown in FIG. 2, when the driver indicates parking intention through the user interface 204, the controller 212 in the ECU 202 sends a light source output control signal to the stripe generator 206 to generate stripe light. Here, the stripe light refers to a light that is approximated to a plane when it is emitted and spread from a light source, and represents light when the light hits an object, but is not necessarily limited thereto.

Immediately after the stripe light occurs, the control unit 212 sends an image acquisition control signal to the camera 208 to acquire the image when the light source is turned on, and then controls the stripe generator 206 to be turned off so that the camera 208 turns off the light source. Acquire an image of the case.

After acquiring the two types of images as described above, the image processing unit 214 obtains the stripe image from the difference between the two images and sends it to the control unit. The control unit 212 analyzes this, sets the parking position, and sends it to the parking assistance controller. Assist in parking your car safely. Meanwhile, a method of setting the parking position using the stripe image will be described later with reference to FIG. 4.

3 is a diagram illustrating a process of obtaining a stripe image in a parking assistance system according to an exemplary embodiment of the present invention.

As shown in FIG. 3, the stripe light is generated at the stripe generator 206 by the light source output control signal from the control unit 212 of the ECU, and the light source in the cell of the image sensor of the camera 208 is generated by the image acquisition control signal. Acquire an image of the lit state. The image processor 214 obtains a stripe image from the difference between the two images.

4 is a diagram illustrating a camera and a stripe generator installed in a parking assistance system according to an exemplary embodiment of the present invention.

As shown in FIG. 4, the stripe generator 206 is installed at the rear left and right centers of the vehicle to emit the stripe light 402 toward the rear of the vehicle. In addition, the vehicle is installed at a point away from the stripe generator 206 by a predetermined distance for photographing the rear of the vehicle. The reason for being installed at a certain distance apart is to recognize the perspective of the vehicle parked behind. Here, for convenience of description, the vehicle is reversely parked by way of example, but the technical matters of the present invention are equally applied to the forward parking.

In addition, the present invention is not limited to the case where the stripe generator 206 is located on the waterline lowered from the camera 208 to the ground, and the stripe generator 206 protrudes more than the camera or enters the vehicle more than the camera. Generally applicable to all mountings in position. This provides mounting ease in determining where to mount the camera 208 and stripe generator 206 of the vehicle.

In addition, the present invention not only considers a case in which the stripe light 402 is emitted parallel to the ground, but also considers a case in which a predetermined angle is inclined with the ground.

That is, the stripe light 402 has a plane shape when it is emitted and has a shape of a fan plane 406 that forms an angle when the emitted stripe light 402 is viewed from above the vehicle. In the present invention, even when installed to have an angle as much as α degrees of the Y axis, the present invention can recognize the three-dimensional parking lot position and also the case where the fan-shaped plane 406 is inclined by ρ degree around the camera Z axis. 3D parking lot location recognition is possible. Details of the 3D parking lot location recognition will be described with reference to FIG. 5.

FIG. 5 is a diagram illustrating a relationship between a plane formed by XYZ coordinates and a stripe light having an origin of a camera installed in a parking assistance system according to an exemplary embodiment of the present invention.

The stripe shape 508 constituting the point P has a shape in which the stripe light is reflected by touching the object, and the image cell of the camera detects the shape and the image plane 504 exists at a distance f of the focal length.

The origin O of the XYZ coordinates shown in FIG. 5 means the position where the image sensor cell of the camera 208 is located, and P ₀ means the point where the light plane 502 formed by the stripe light meets the Y axis.

As shown in FIG. 5, the normal vector (0,1,0) of the XZ plane is rotated by ρ degrees with respect to the Z axis and by (π / 2-α) degrees with respect to the X axis. Find the normal vector n of (502).

The equation of the optical plane 502 can be obtained from [Equation 2] using the lawsen vector of [Equation 1] and a point P ₀ on the optical plane.

The origin O of the XYZ coordinates, one point p (x, y) on the xy plane, which is the image plane 504 recognized by the image cell of the camera, and one point P (X, Y, Z) on the optical plane 502, are all straight lines Q. Above 506. Using the parameter k can be expressed as [Equation 3]. Where f is the focal length of the camera.

Therefore, the straight line Q506 is calculated as shown in [Equation 4].

Since a point P in the band shape 508 is an intersection point of the optical plane 502 and the straight line Q 506, the following equations (2) and (4) are satisfied. In this case, substituting [Equation 4] and [Equation 1] into [Equation 2]

If we solve for k, we get [Equation 5].

Substituting [Equation 5] into [Equation 4] to obtain the X, Y and Z coordinates of the point P in the band shape 508, Equation 6, Equation 7 and Equation 8 are obtained, respectively. Comes out.

To recognize the three-dimensional position of the ground using P (X, Y, Z) obtained as described above, it must be converted to the car coordinate system P '(X', Y ', Z').

The car coordinate system is defined such that the foot of the waterline relative to the ground surface of coordinate O in FIG. 5 is the origin and the X'Z 'plane is parallel to the ground surface.

A point P on the camera coordinate system is rotated by γ degrees with respect to the X axis and moved by the camera height c in the Y axis direction to obtain a point P 'on the vehicle coordinate system as shown in [Equation 9].

By calculating in this way, a bundle of distance data in which a stripe of light strikes an object and the image detected by the camera has a band shape can be obtained.

On the other hand, the camera uses a large field of view (FOV) lens (ex. 120 degrees) to capture a large space.

FIG. 6 is a diagram illustrating a stripe image obtained by the parking assisting system according to an exemplary embodiment of the present invention.

As shown in FIG. 6, FIGS. 6A, 6B and 6C show images obtained with the light source turned on, and FIG. 6B shows images obtained with the light source turned off and the difference image, respectively.

FIG. 7 is a diagram illustrating a stripe image as data of an XZ plane of a car coordinate system in a parking assistance system according to an exemplary embodiment of the present invention.

As shown in FIG. 7, the stripe image is represented by a set of points on the XZ plane.

8 is a diagram illustrating a method of detecting an empty space using stripe data in a parking assistance system according to an exemplary embodiment of the present invention.

As shown in FIG. 8, two three-dimensional points that exist in adjacent columns on the stripe image and are farther than a threshold (eg, length of the magnetic field) on the XZ plane are detected as occlusion as shown in FIG. 8. do. Occlusion has the property of whether it is the left end or the right end of an object. The occlusion at the left end of the object is marked with an 'x', and the occlusion at the right end of the object is marked with an 'o'. At this time, the left end occlusion of the object and the right end occlusion of the object exist in pairs. If the left end occlusion of an object in the occlusion pair is closer to the camera than the right end occlusion of the object, then the occlusion pair is found at the left end of the object of interest (objects to the left and right of the empty space). Can be. If the left end occlusion of the object of the occlusion pair is farther from the camera than the right end occlusion of the object, it can be seen that the occlusion pair is present on the right side of the object of interest. This property is recorded as direction information of the occlusion pair.

In the occlusion pair, the occlusion that satisfies the following three conditions is recognized as a pivot point.

First, in the direction of the occlusion pair (eg a semicircle with the occlusion closest to the camera among the occlusion pairs, the width of the deviation as the radius, and the line segments between the occlusion pairs as sides). There is an empty space.

Second, there is sufficient length (e.g. width of the vehicle) to the field of view (FOV) boundary in the direction of the occlusion pair.

Third, the occlusion closest to one camera closest to the optical axis among the occlusion pairs satisfying the above conditions is recognized as a pivot.

It can be seen that the pivot that satisfies the above condition is shown in FIG. 8.

9 is a diagram illustrating a method of recognizing a reference point opposite to a region of interest in a parking assistance system according to a preferred embodiment of the present invention.

 As shown in FIG. 9, points separated from the pivot in a direction of the pivot (the direction of the occlusion pair to which the pivot belongs) by a predetermined distance (e.g. the length of the vehicle) are recognized as distance data of the opposite obstacle. Red dots are indicated in the figure below. Recognize the closest distance data from the double pivot (indicated by 'o') as the opposite reference point. Recognized opposite reference points are marked with 'x'.

 The side direction of the opposite object is recognized by using distance data belonging to a certain distance (e.g. 2/3 of the length of the vehicle) in the direction away from the camera while being perpendicular to the pivot direction at the opposite reference point. In the figure below, the sides of the opposite object were recognized using the distance data in yellow. The side of the recognized object was marked with a blue line.

10 is a view illustrating a target parking position set in the parking assistance system according to the preferred embodiment of the present invention.

 From the center of the pivot and the opposing reference point, set the rectangle with the width of the vehicle and the length of the vehicle in the direction of the side of the object opposite to the target parking position. The red square in the figure below shows the set parking position.

11 is a flowchart illustrating a parking position setting method of a vehicle according to an exemplary embodiment of the present invention.

As shown in FIG. 11, the stripe generator is first turned on (S1102) and an image is acquired from the camera (S1104). Next, the stripe generator is turned off (S1106) and the image with the light source turned off is obtained (S1108).

In operation S1110, a stripe image is obtained from a difference between two images obtained when the stripe generator is turned on and off. The stripe image is analyzed by analyzing the obtained stripe image and the space parking space is extracted (S1112), and the space is transmitted to the parking assistant controller (S1114).

12 is a flowchart illustrating a pedestrian warning method during vehicle parking according to a preferred embodiment of the present invention.

As shown in FIG. 12, when parking starts, first, the stripe generator is turned on (S1202) and an image is acquired from the camera (S1204). Next, the stripe generator is turned off (S1206) and the image with the light source turned off is obtained (S1208).

In operation S1210, a stripe image is obtained from a difference between two images obtained when the stripe generator is turned on and off. The stripe image is analyzed and the distance data bundle is extracted (S1212), and it is determined whether a distance data bundle more than a threshold is generated (S1214). At this time, if there is a bundle of distance data above the threshold value, it is determined that the pedestrian appears and notifies the parking assistance controller that the pedestrian is detected (S1216).

If there is no bundle of distance data above the threshold, the process proceeds to step S1202 again to obtain an image of the stripe generator on and off and to repeatedly check whether a pedestrian exists.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

According to the present invention, when parking a vehicle, it is possible to recognize a parking available position even in a dark parking lot, thereby increasing the driver's parking convenience and alerting when a pedestrian appears on the parking path while parking to the recognized parking position. It is effective to increase.

1 is a block diagram showing a semi-automatic parking assistance system to which the parking assistance system of the present invention is applied;

2 is a view showing a parking assistance system according to a preferred embodiment of the present invention,

3 is a diagram illustrating a process of obtaining a stripe image in a parking assist system according to an exemplary embodiment of the present invention;

4 is a view illustrating a state in which a camera and a stripe generator are installed in a parking assistance system according to a preferred embodiment of the present invention;

5 is a view showing a relationship between a plane formed by X, Y, Z coordinates and a stripe light having a camera installed in a parking assistance system according to a preferred embodiment of the present invention;

6 is a view showing a stripe image obtained in a parking assist system according to a preferred embodiment of the present invention;

7 is a view showing a stripe image as data of an XZ plane of a car coordinate system in a parking assistance system according to a preferred embodiment of the present invention;

8 is a diagram illustrating a method of detecting a region of interest using stripe data in a parking assistance system according to an exemplary embodiment of the present invention;

9 is a diagram illustrating a method for recognizing a reference point opposite to a region of interest in a parking assistance system according to a preferred embodiment of the present invention;

10 is a view showing a target parking position set in the parking assistance system according to a preferred embodiment of the present invention;

11 is a flowchart illustrating a method for setting a parking position of a vehicle according to an exemplary embodiment of the present invention.

12 is a flowchart illustrating a pedestrian warning method during vehicle parking according to a preferred embodiment of the present invention.

<Description of Symbols for Main Parts of Drawings>

102: parking lot recognition system 104; User interface

106: parking assistance controller 108: vehicle attitude sensor

110: active steering system 202: ECU

204: user interface 206: stripe generator

208: camera 212: control unit

214: Image processing unit 402: Striped light

406: flat fan 502: light plane

504: image plane 506: straight line Q

508: strip shape

Claims (11)

In the parking assistance system, A user interface for informing a driver of the vehicle of information including a surrounding image of the vehicle; A camera for photographing the surrounding image of the vehicle; A stripe generator for generating stripe light around the vehicle; And Turn the stripe generator on or off and send a signal to the camera to take a picture of the surrounding image, and then analyze the surrounding image acquired by the camera to find the parking position of the vehicle and the vehicle to the parking position. An electronic control unit (ECU) that alerts through the user interface when a pedestrian is detected on a parking path while parking. Parking assistance system comprising a. The method of claim 1, wherein the electronic control device, And an error due to an angle formed by a plane formed by the stripe light emitted from the stripe generator to the ground when the rear image obtained by the camera is analyzed. The method of claim 1, wherein the stripe generator, And a plane formed by the stripe light emitted at an angle to the ground. The method of claim 1, wherein the stripe generator, Parking assistance system, characterized in that for generating light of the wavelength of the infrared region. The method of claim 1, wherein the camera, Parking assist system, characterized in that for each of the stripe generator is turned on and the stripe generator is turned off to send the captured image to the ECU. In the parking assistance method, (a) acquiring an image when the stripe generator mounted on the vehicle is turned on and an image when the stripe generator is turned off; (b) obtaining a stripe image from a difference between an image when the stripe generator is turned on and an image when the stripe generator is turned off; (c) obtaining distance data from the stripe image by correcting an error due to an angle between a plane formed by the stripe light emitted from the stripe generator and an angle with the ground; (d) recognizing the parking position from the distance data; (e) recognizing that parking is started to the parking position; (f) obtaining an image of turning on the stripe generator and an image of turning off the stripe generator; (g) obtaining a stripe image from a difference between an image of turning on the stripe generator and an image of turning off the stripe generator; (h) obtaining distance data from the stripe image by correcting an error due to an angle between a plane formed by the stripe light emitted from the stripe generator and the ground; (i) determining whether distance data above a threshold exists; (j) proceeding to step (l) if there are distance data above a threshold value as a result of the determination of step (i); (k) proceeding to step (f) if there is no distance data above a threshold as a result of the determination of step (i); And (l) notifying that the pedestrian is detected by the user interface mounted on the vehicle; Parking assistance method comprising a. The method of claim 6, wherein step (a) is And photographing an image in which the stripe generator is turned on, and then obtaining an image of turning off the stripe generator. In the parking position setting method of the vehicle, (a) acquiring an image of turning on the stripe generator mounted on the vehicle and an image of turning off the stripe generator; (b) obtaining a stripe image from a difference between an image when the stripe generator is turned on and an image when the stripe generator is turned off; (c) obtaining distance data from the stripe image by correcting an error due to an angle between a plane formed by the stripe light emitted from the stripe generator and an angle with the ground; And (d) recognizing the parking position from the distance data Parking position setting method of the vehicle comprising a. The method of claim 8, wherein step (a) comprises And photographing an image of turning on the stripe generator and obtaining an image of turning off the stripe generator. In the vehicle pedestrian warning method, (a) acquiring an image of turning on the stripe generator mounted on the vehicle and an image of turning off the stripe generator; (b) obtaining a stripe image from a difference between an image when the stripe generator is turned on and an image when the stripe generator is turned off; (c) obtaining distance data from the stripe image by correcting an error due to an angle between a plane formed by the stripe light emitted from the stripe generator and an angle with the ground; (d) determining whether there is a distance data above a threshold; (e) proceeding to step (g) if there are distance data above the threshold as a result of the determination of step (d); (f) proceeding to step (a) if there is no distance data above the threshold as a result of the determination of step (d); And (g) notifying that the pedestrian is detected by the user interface mounted on the vehicle; Pedestrian warning method during vehicle parking, comprising a. The method of claim 10, wherein step (a) comprises And taking an image of turning on the stripe generator and obtaining an image of turning off the stripe generator.

Images