KR102485480B1 - A method and apparatus of assisting parking by creating virtual parking lines - Google Patents

Abstract

A parking assist device according to an aspect of the present invention acquires an image of the surroundings of a vehicle using a camera, and an AVM system unit that generates a top-view image using the acquired image, and an edge image from the top-view image. An image processing unit that generates a parking division line calculation unit that calculates a parking division line from the top view image and the edge image, a virtual parking reference line generation unit that generates a virtual parking reference line from the parking division line, and based on the parking division line and the virtual parking reference line and a parking target area determining unit that determines a parking target area.

Description

Parking assisting device and method by creating virtual parking lines {A method and apparatus of assisting parking by creating virtual parking lines}

The present invention uses an around view monitoring system (hereinafter referred to as 'AVM system') and a smart parking assist system (SPAS) to recognize a parking target area and provide a moving path to the driver. It relates to a supporting device and method thereof.

Recently, various inventions related to parking assistance devices have been filed.

Conventionally, when there is a parked vehicle, an ultrasonic sensor is used to determine the position of an obstacle such as a vehicle to calculate a parking target area. There was a problem that it could not be used if the vehicle was not available.

In addition, when there is no parked vehicle, an invention capable of recognizing a parking line of a parking lot using a camera and calculating a parking available area is actively being invented. An AVM system that creates a flat screen from a camera image, and a SPAS system that creates a parking movement path using an ultrasonic sensor and automatically parks, are being developed.

A parking assist device using a conventional AVM system recognizes a parking reference line (parallel to the vehicle's driving direction), which is the entrance of a parking zone, and calculates a parking zone (perpendicular to the vehicle's driving direction or at a certain angle) based on the parking reference line, thereby providing a parking target. determine the area According to the prior art, there is no problem in the case of a closed type parking section in which a parking reference line is displayed, but in the case of an open type parking section without a parking reference line, it is difficult to accurately recognize a parking target area.

In order to solve the above problems, an object of the present invention is to provide a method for generating a virtual parking line, recognizing and providing a parking target area to a user even in the case of an open parking section.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the description below.

A parking assist device according to an aspect of the present invention for achieving the above object is an AVM system unit that obtains an image around a vehicle using a camera and generates a top-view image using the acquired image. An image processing unit generating an edge image from the top-view image, a parking division line calculation unit calculating a parking division line from the top-view image and the edge image, a virtual parking reference line generation unit generating a virtual parking reference line from the parking division line, and the parking division line and a parking target area determination unit determining a parking target area based on the virtual parking reference line.

A parking assistance method according to another aspect of the present invention includes acquiring images captured using one or more cameras, generating a top-view image by combining the acquired images into a plan view, and converting the top-view image into an edge image. step of scanning the road surface in the vehicle traveling direction, detecting feature points of a bright color system, recognizing a boundary line from a boundary point around the detected feature point, and calculating a parking division line from the boundary line, from the detected parking division line calculating parking section intersection points, connecting the parking section intersection points to create a virtual parking reference line, and determining a parking target area from the parking section intersection line and the virtual parking reference line.

According to the present invention, even in the case of an open parking area, there is an effect of clearly recognizing a parking target area by generating a virtual parking line.

1 is a diagram illustrating a top-view image generated by an AVM system;
2a is a schematic diagram of a closed orthogonal parking compartment;
Figure 2b is a schematic diagram of an open right angle parking compartment;
Fig. 2c is a schematic diagram of a closed oblique parking compartment;
2d is a schematic diagram of a closed oblique parking compartment;
3 is a form of a top-hat filter used for feature point search.
4A is a diagram for explaining a method of calculating a parking section line in an open type right-angle parking section.
4B is a diagram for explaining a method of generating a virtual parking reference line in an open type perpendicular parking section;
4C is a view for explaining a method of calculating a parking division line in an open oblique parking division;
4D is a diagram for explaining a method of generating a virtual parking reference line in an open oblique parking section;
5A is a flow chart illustrating a method for calculating a parking division line;
5B is a process flow diagram illustrating a method of generating a virtual parking reference line;
6A is a diagram for explaining a method of generating a forward movement path and controlling a steering wheel in a right-angled water parking section;
6B is a view for explaining a method of generating a reverse movement path and controlling a steering wheel in a perpendicular vertical parking section;
7A is a view for explaining a method of reverse parking in an oblique parking area;
7B is a view for explaining a method of forward parking in an oblique parking area;
8 is a flowchart of a parking assistance method by generating a virtual parking reference line;

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Meanwhile, terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" means that a stated component, step, operation, and/or element is the presence of one or more other components, steps, operations, and/or elements. or do not rule out additions.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 shows a diagram showing a top-view image of an AVM system.

An AVM system (Around view monitoring system) refers to a system that acquires images in all directions using one or more cameras, and synthesizes the acquired images into one screen to provide to a user.

The camera of the present invention can create a composite screen using one or more cameras as needed. For example, a total of four cameras can be installed, one for each front, back, left, and right side, or one 360-degree camera can replace it. Images obtained from one or more cameras are synthesized as shown in FIG. 1 .

A normal AVM system synthesizes and transforms a plurality of screens taken from the front, rear, left, and right sides to create a top-view image (composite image) 100 that looks like a plan view viewed from above. Conventionally, it was difficult to recognize a parking line directly from a camera image, but the AVM system has been improved and it has become easy to calculate a straight-line parking line using the top-view image 100 in the form of a plan view.

The AVM system has a problem in that it is difficult to obtain an accurate image when the lane is blocked by a parked vehicle and the boundary of the synthesized image is distorted, but has the advantage of being able to view images in all directions at a glance.

The synthesized image generated by the AVM system looks like a photograph taken from the top of a car, so it is called a top-view image or an around view image.

In the present invention, the top-view image 100 is displayed to the user, and the parking area 600 is determined by calculating the parking division line 220 using the information analyzed from the top-view image 100, and the SPAS (Smart parking assist system) ) calculates a movement path using obstacle information such as a parked vehicle recognized by an ultrasonic sensor and the top-view image 100, and controls the steering wheel to automatically park.

In addition to the top-view image 100 generated by the AVM system, the present invention can utilize obstacle information using an ultrasonic sensor and information using v2x technology.

v2x technology is an abbreviation of vehicle to everything. It is a technology that can exchange information between a car and other devices through communication means, specifically v2d (vehicle to device), v2v (vehicle to vehicle), v2g (vehicle to grid) v2i (vehicle to infrastructure).

In the present invention, v2v technology capable of receiving information obtained during pre-parking from a v2x module installed in another parked vehicle can be utilized, and when receiving information inside a parking lot from a v2x module installed in a parking lot, parking is parked regardless of the type of parking section. You will be able to quickly determine the target area.

Figure 2a shows a schematic diagram of a closed orthogonal parking compartment.

Figure 2b shows a schematic diagram of an open orthogonal parking compartment.

Figure 2c shows a schematic diagram of a closed oblique parking compartment.

Figure 2d shows a schematic diagram of an open oblique parking compartment.

A conventional method for determining a parking target area is a method of first detecting a parking reference line 210 shown in FIG. 2A or 2C and then detecting a parking division line 220 that is perpendicular or at a certain angle from the detected parking reference line 210. use. In the case of the closed type parking section as shown in FIGS. 2A and 2C, since the parking reference line is displayed, a conventional method of recognizing the target parking area can be used. However, in the case of the open parking section as shown in FIGS. 2B and 2D, the parking reference line does not exist. Therefore, there is a problem in that a conventional method for recognizing a parking target area cannot be used.

In order to solve the above problem, the present invention is a method of first calculating the parking division line 220 and generating a virtual parking reference line 450 using the parking division intersection 440 included in the calculated parking division line 220. Use For detailed description, refer to FIGS. 4A to 4D.

3 shows a top-hat filter used for detecting the feature point 300 and a schematic diagram showing the position of the feature point 300.

The parking division line 220 generally has a brighter color compared to the road surface, and a point having a brighter color than the surroundings can be searched from the top-view image generated from the AVM system. In the case of a perpendicular parking area, since the parking division line 220 exists perpendicular to the direction in which the vehicle travels or in a direction at a constant angle, after removing noise from the top-view image, in a direction parallel to the direction of travel of the vehicle (FIG. 4a or FIG. According to the feature point search direction of 4c), the approximate position of the feature point 300 of the parking dividing line 220 is searched by applying a top-hat filter. A detailed search method for calculating the parking division line 220 from the feature points 300 will be described with reference to FIG. 4A.

In the open type perpendicular parking section, FIG. 4A is a diagram for explaining a method of calculating the parking section line 220, and FIG. 4B is a diagram for explaining a method for generating a virtual parking reference line 450.

As described in FIG. 3, the approximate position of the feature point 300 is searched from the vehicle traveling direction in the top-view image 100, and the reference boundary points on both sides of the feature point 300 are searched by using a local maxima search technique. (410) is detected.

The top view image 100 is an edge image (by methods such as Sobel edge detection, Prewitt edge detection, Roberts edge detection, and Canny edge detection). Edge image, an image obtained by converting the boundary of the image to the boundary point 415).

Sobel edge detection is the ratio of the sum of the absolute values (|G|=|Gx|+|Gy|) and the change (θ=arctan(|Gy) after obtaining the horizontal brightness change (Gx) and vertical brightness change (Gy) /Gx|) If |G| is below the threshold, it is processed as noise, and if it is above the threshold, it is processed as an edge fixel (415).

Each edge detection method has a different mask used to measure the amount of change in brightness, and has advantages and disadvantages in operation speed, accuracy, and edge detection degree.

In the case of conversion to an edge image, even in the same parking lot, the edge detection result may be different depending on the external illumination. In the case of outdoors during the day when there is a lot of light, the external illumination may be high, and during the night time, the external illumination may be low. Even in the case of an underground parking lot, external illuminance may be different depending on whether an external light source exists.

If the location of the parking lot can be identified using v2x technology, the brightness of a specific point can be measured and used as a standard for external illumination. Sensitivity can be set high or low by changing a mask value to be used for edge detection according to the measured external illumination.

A mask means a matrix used in the edge detection method, and is used to calculate a brightness change parameter around a point to measure brightness using a 2x2 matrix, a 3x3 matrix, a 4x4 matrix, a 5x5 matrix, and the like.

If it is impossible to compare the brightness of a specific point because the parking lot visited for the first time or the location of the parking lot cannot be identified, a method of measuring external illumination using an image taken at a specific point outside the vehicle body (for example, the bonnet) is used.

For example, in a dark environment, since the amount of light reflected on the bonnet is small, it is determined that external illumination is low accordingly, and a mask more sensitive to changes in brightness is used. On the other hand, even when the external illumination is high during the day, a mask that is more sensitive to changes in brightness is used. However, in a dark case, it is preferable to increase the sensitivity only for pixels corresponding to a dark brightness, and in a bright case, increase the sensitivity only for pixels corresponding to a bright brightness. In the method of selectively applying sensitivity, the mask to be used is changed based on the average brightness of the pixels being used. For example, in the case of using a 3x3 matrix, when measuring, the average of the brightness of the pixels in the eight directions and the center pixel is obtained, and when the brightness is higher than the first threshold for brightness, a mask for bright pixels is used, and a mask for the bright pixels is used and the brightness is below the second threshold for brightness. If it is dark, use a mask for dark pixels. Here, the first threshold for brightness is greater (brighter) than the second threshold.

New boundary points 415 are detected based on both reference boundary points 410 near the feature point 300 in the edge image. A boundary line may be estimated using an extended Hough transform of the detected boundary points 415 . An optimal boundary line is estimated by repeating rotation around the reference boundary point 410 by 1 degree.

Since there are two reference boundary points 410 on both sides of the feature point 300, the boundary line is estimated twice for one feature point 300, and as a result, the estimated boundary line is called the first boundary line and the second boundary line. call The first boundary line and the second boundary line are likely to be both boundaries of the parking division line 220 .

Since the width of the parking division line 220 is predicted to be constant, if the slope and distance between the estimated first boundary line and the second boundary line are values within a threshold value, the first candidate parking division line is determined using the average slope and the feature point 300. extract At this time, the thickness of the line of the first candidate parking division line (distance between the first boundary line and the second boundary line) is stored together using the angle and distance between the first boundary line and the second boundary line.

The distance between the first boundary line and the second boundary line is measured in a direction perpendicular to the average slope of both boundary lines and an average value is used, or a maximum value and a minimum value are measured and averaged. In general, it can be said that the thickness of the parking section line is constant, so the difference in calculation has little effect on the result.

A first parking division line candidate is extracted for each feature point 300, and a second parking division line candidate is extracted by comparing the slope and interval between the different first parking division line candidates and the line thickness of each first candidate.

Since the thickness of the parking division lines 220, the slope between the parking division lines 220, and the interval between adjacent parking division lines 220 will be constant, the thickness of the line among the first candidates for the parking division lines, the distance between the first candidates for the parking division lines, and the distance between the adjacent first candidates When the slope of the first candidate has a value within a certain range, it is determined as the second candidate for the parking division line.

The method of comparing the slopes to extract the second parking division line candidate is to obtain the average slope of the first parking division line candidates, determine the first parking division line candidate having the slope closest to the average slope as the representative parking candidate line, and determine the other parking division line candidates. The difference between the slope of the first candidate and the slope of the representative line of the parking candidate is compared. This method is useful for removing minor noise.

The parking candidate representative line can be recalculated taking into account the excluded candidates. If there are a plurality of first parking division line candidates having completely different directions, the first parking candidate representative line may be inaccurate, so the parking candidate representative line is recalculated after excluding the first parking division line candidates having a large difference from the slope of the parking candidate representative line. Thus, the second candidate for the parking division line can be calculated.

When the problem cannot be solved because the number of candidates having different slopes is similar, the slope of the representative line of parking candidates may be selected as a weighted average value obtained by weighting the accuracy of the boundary line of the edge image.

The accuracy of the boundary line is the difference between the slopes of the first boundary line and the second boundary line, the distance between the first boundary line and the second boundary line, and the standard error (e.g., residual sum of squares, residual sum of squares) in regression analysis when estimating the first boundary line from the boundary points. It is calculated by applying weights based on squares).

Another method of comparing the slopes to extract the second candidate parking division line is to sort based on the slope of the first candidate parking division line, determine the slope threshold range for determining that the slopes are the same, and determine the slope threshold for each of the first candidate parking division lines. The number of other parking division line first candidates included in the range is counted, and the parking division line first candidate including the largest number of parking division line candidates within the gradient threshold range is determined as the parking candidate representative line, and the gradient of the parking candidate representative line is within the gradient threshold range. All of the first parking division line candidates included may be determined as the second parking division line candidates.

When the slope of the first candidate parking division line is tan(θ), it is preferable to set the slope threshold range based on the angle (θ).

At this time, considering the total number of candidates belonging to the first candidate group, if the difference between the minimum value and the maximum value of the slope is within twice the threshold threshold range of the slope, all the first parking division line candidates are assigned to the second parking division line without the above sorting and determination processes. It is determined that the gradient condition of the candidate is satisfied, and there is no need to determine the representative line of the parking candidate.

Since the interval between adjacent parking division lines 220 of a parking lot is often constant, another condition for extracting the second parking division line candidate is whether the interval between adjacent first candidates is included within a certain range.

When the interval between adjacent second candidates is close to an integer multiple of 2 or more of the interval between other adjacent second candidates, the parking division line 220 in the middle is erased due to weathering, etc. Determined as the second candidate parking division line Since there is a possibility that the parking division line 220 is predicted to exist, the first parking division line candidate is precisely re-detected at the location where the parking division line 220 is predicted to exist, and then it is determined whether the parking division line is a second candidate.

The parking division line 220 is calculated when the distance between adjacent second candidates among the second candidate group of parking division lines, the direction of the second candidate, and the thickness of the line satisfy the law. Information such as the thickness of the parking division line 220, the distance between the parking division lines 220 on both sides, and whether the area to be parked for the vehicle is on the left side or the right side is stored together as the parking division line 220 information.

Among two or more parking division lines 220, two adjacent parking division lines 220 may be selected to determine the final parking target area 600.

The type of parking section is determined from the direction and interval of the calculated parking section line 220 . In the case of a closed type parking area, when the angle formed by the parking reference line 210 and the parking dividing line 220 is 90 degrees, it is determined as a vertical parking area or a horizontal parking area. The distinction between the vertical parking area and the horizontal parking area can be seen from the length of the parking dividing line 220 and the distance between the parking dividing lines 220 . If the angle formed by the parking reference line 210 and the parking division line 220 is not 90 degrees, for example, 30 degrees, 45 degrees, or 60 degrees, it is determined as an oblique parking area.

In the case of a closed parking area, the parking target area 600 may be determined by first detecting the parking reference line 210 and then detecting the parking division line 220 based on the parking reference line 210 according to the prior art. According to this, the virtual parking reference line 450 may be detected and the actual parking reference line 210 may be prepared.

A preferred embodiment of determining the parking target area 600 according to the present invention is

1) If there is a parked vehicle, an appropriate parking area is calculated by identifying obstacles using an ultrasonic sensor. When a parked vehicle exists, the parking area may be extracted only with the vehicle since the parking line may be difficult to recognize because it is covered by the vehicle. However, even when a parked vehicle exists, if the parking division line 220 or the parking reference line 210 is clearly recognized, the analysis result of the top-view image 100 from the AVM system and the obstacle information identified using the ultrasonic sensor are used together in the parking area It is desirable to calculate

2) In the case of a closed parking area where no parked vehicles exist, it is preferable to first calculate the parking reference line 210 and calculate the parking division line 220 based on the parking reference line 210, but virtual parking of the present invention The method by generating the reference line 450 can be used simultaneously.

3) In the case of an open parking area without a parking reference line 210, the parking target area 600 is determined by first detecting the parking division line 220 and then generating a virtual parking reference line 450.

Depending on the given parking situation, it is desirable to combine the prior art and the technical idea of the present invention.

In the case of an open parking area where no parked vehicle exists, the parking division line 220 is first detected as described above, and the virtual parking reference line 450 is estimated from the parking division line 220 by the method described in FIG. 4B below. do. A perpendicular parking area, a parallel parking area, and an oblique parking area are determined based on an angle formed between the parking division line 220 and the virtual parking reference line 450 .

FIG. 4B is an exemplary view for explaining a method of generating a virtual parking reference line 450 from a parking division line 220. Referring to FIG.

An end point is extracted from each parking division line 220 determined in FIG. 4A. The extracted end point is referred to as a parking section intersection 440 . A virtual parking reference line 450 is determined by connecting two or more extracted intersection points. The intersection is a point where the virtual parking reference line 450 and the parking division line 220 intersect.

At this time, the position of the intersection may be adjusted using the position and direction of the parked vehicle by the ultrasonic sensor. This is because the length of the parking section line 220 may be unreliable, such as the end part being erased or damaged, so it is necessary to conservatively set the parking section intersection 440 to prevent accidents.

When three or more parking section intersections 440 exist, the virtual parking reference line 450 is determined through regression analysis based on the least square error method. The width of the virtual parking reference line 450 may be 0 and may be determined based on the average width of the parking division line 220 .

When the virtual parking reference line 450 is determined, the smart parking assistance system operates based on the virtual parking reference line 450 . When moving forward for perpendicular parking in reverse, the driving direction of the vehicle may be determined parallel to the virtual parking reference line 450 . A detailed description will be given in FIGS. 6A and 6B.

The detected parking division line 220 may be re-verified by re-extracting the parking division line 220 based on the virtual parking reference line 450 and comparing the extracted parking division line 220 with the previously extracted parking division line 220 . In the method of extracting the parking division line 220 using the feature point 300, noise can be determined as the feature point 300.

Using the virtual parking reference line 450 generated by appropriate estimation and the prior art, it is possible to more accurately predict the parking division line 220, and the predicted parking division line 220 is different from the previously extracted parking division line 220 If so, since the virtual parking reference line 450 may be an incorrectly set result, another parking area may be found.

So far, a method of extracting a parking segment 220 based on a single image has been described. When actually parking, the top view image 100 is continuously generated because it is moving in the parking lot, and the dynamic system model of the Kalman filter is used to measure and correct the parking division line 220 based on the continuous images.

With the recent development of the AVM system, it is possible to create a top-view image 100 that synthesizes and converts the image of the camera to accurately display the floor plane of the parking lot. Therefore, when the vehicle approaches the parking division line 220 to be detected, the parking division line 220 can be more accurately calculated.

When the parking division line 220 is calculated based on the continuous image, the parking division line 220 is repeatedly calculated at regular time intervals. If the time interval is defined as t0, t1, t2, etc. , The parking division line 220 of the present (period t0) is measured, and the parking division line 220 to be measured in the near future (period t1) based on the information (moving speed and direction of movement) of the period t0 can be predicted at the time point t0 there is.

The parking segment line 220 is measured again at period t1, and when the predicted value of the previous period (t0) and the measured value of the current period (t1) are different, the parking segment line 220 of period t2 is predicted using the inductively adjusted prediction parameter do. This method is called the dynamic system model of the Kalman filter.

There are many cases where there is a main parking lot used by each user, and the parking environment can be different for each user. Therefore, it is more efficient to store necessary information for each parking and use it for subsequent parking rather than using a uniform prediction parameter. . Particularly, in the case of continuous images, it may be desirable to make predictions by different prediction parameters for a parking area located far away from a parking area located near a distance.

For example, if a driver commutes by car, the parking division line 220 is calculated for the parking lot of the company and the parking lot of the house, input information and output information for determining the parking target area 600 are stored in the form of a template, and input After identifying the characteristics of information and making an index, when the same information is received later, output information can be retrieved from past data without new calculations, and necessary information can be created by reflecting only other information (parked vehicles, obstacles, etc.) There will be. Using the template, it is possible to increase the accuracy of the parking target area determination and parking assistance method while reducing calculation time.

Information (position, thickness, spacing, direction, etc.) ) together.

According to the present invention, since the parking division line 220, the virtual parking reference line 450, and the parking target area 600 can be quickly calculated, a forward movement path is generated by avoiding obstacles in order to park in the parking target area 600. While moving by controlling the steering wheel, forward movement path and backward movement path are regenerated based on updated data according to the Kalman filter method.

FIG. 4C is a diagram for explaining a method of calculating a parking division line 220 in an open oblique parking section, and FIG. 4D is a diagram for explaining a method for generating a virtual parking reference line 450 in an open oblique parking section. to be.

In the oblique parking area, as in the perpendicular parking area, (1) search for the feature point 300, (2) detect the boundary point 415, (3) estimate the boundary line, (4) extract the first candidate for the parking section line, , (5) extracting the second candidate parking division line, (6) calculating the parking division line 220, (7) generating the virtual parking reference line 450, and (8) determining the parking target area 600. Follow the order.

The difference between the two is due to the different angles in the step of estimating the boundary line. For example, the angle may be 30 degrees, 45 degrees, 60 degrees, etc. based on the traveling direction of the vehicle.

5A shows a procedure flow diagram explaining a method of calculating a parking segment 220.

The method of detecting the parking section line 220 according to the present invention (1) since processing all pixels of a 2D image takes a long time, one-dimensionally searching for a plurality of feature points 300 (S510) , (2) detecting an edge fixel 410 near the detected feature point 300 (S521), (3) searching for a direction in units of 1 degree based on the reference edge point 410, and (4) extracting a pair of boundary lines from the extracted boundary points 415 (S522), and extracting a first parking division line candidate whose slope difference between the two boundary lines is within a threshold value (S531) , (5) extracting a second candidate from the first candidate in which the slope and spacing between candidates fall within a certain range (S532), (6) based on the spacing and angle of the parking line according to the law from the second candidate group, a single image parking dividing line (220) ) is calculated (S540), and (8) the information of the continuous image parking section line 220 is updated using a Kalman filter (550).

5B shows a procedure flow diagram explaining a method of estimating a virtual parking reference line 450 from a detected parking dividing line 220 .

From the calculated parking division line 220, a parking division intersection point 440 as an end point is detected (S560), a virtual parking reference line 450 is estimated from the detected intersection point (S570), and the parking division line 220 and the virtual parking reference line are estimated. The parking target area 600 is determined using (450) (S580).

6A is a view for explaining a method of recognizing a parking target area 600 and obstacles for reverse parking in an open perpendicular parking section according to the present invention, avoiding obstacles, creating a forward movement path, and controlling a steering wheel. indicate 6B shows a view for explaining a method of generating a reverse movement path and controlling a steering wheel in an open perpendicular parking section according to the present invention.

If the virtual parking reference line 450 is estimated, the distance from the virtual parking reference line 450 and the deflection angle are calculated, the direction is corrected so that the deflection angle becomes 0 degrees, and after properly moving forward, the vehicle parks in reverse.

In the case of perpendicular parking spaces, reverse parking is the rule. For reverse parking, a forward movement path is generated in the same direction as the direction of the virtual parking reference line 450, and the driving wheel is controlled to move forward (FIG. 6A). After the forward movement, a backward movement path is created and the steering wheel is controlled to move backward (FIG. 6b).

According to FIG. 6A , if the parked vehicle is parked beyond the virtual parking reference line 450, the steering wheel of the driving vehicle is manipulated to avoid the obstacle (parked vehicle) and move forward. That is, (1) maintain a constant distance between the driving vehicle and the virtual parking reference line 450, (2) maintain parallel to the vertical axis of the virtual parking reference line 450 and the vehicle, and (3) maintain a constant distance between the obstacle and the vehicle. keep moving forward

The prior art moves by controlling the steering wheel after calculating the movement path in a stationary state, but the present invention generates a virtual parking reference line 450 for generating a forward movement path and determines a forward movement path in a state in which a parking target area is determined. can be calculated immediately, and since the movement path generation result can be updated simultaneously while moving, there is an effect of enabling faster parking compared to the prior art.

According to FIG. 6B, the vehicle is parked in reverse in the parking target area 600, and the driving direction (longitudinal axis direction) of the vehicle is parallel to the direction of the virtual parking reference line 450, and a certain distance is maintained from the parked vehicle. The area, horizontal length, and vertical length of the parking target area 600 must be greater than the area, horizontal length, and vertical length of the vehicle.

When parking in reverse, the steering wheel is adjusted so that the longitudinal axis of the vehicle coincides with the vertical center line of the parking target area 600, and a safe distance is maintained so as not to come into contact with the parked vehicle.

Accurately determining the parking target area 600 using the AVM system and the ultrasonic sensor is advantageous in reverse parking because the most information can be acquired when the parking target area 600 is passed.

7A shows an exemplary view for explaining a method of parking in reverse according to the direction of an oblique line in the case of an open oblique parking section according to the present invention.

7B shows an exemplary view for explaining a method of forward parking according to the direction of an oblique line in the case of an open oblique parking section according to the present invention.

In the case of reverse parking according to FIG. 7A, only the angle is different, and the same method as in the case of reverse parking in the method described in FIG. 6B can be used, but the case of forward parking according to FIG. 7B will be different. In the case of an oblique parking area, reverse parking may not be possible because the width of the road is narrow. In addition, it may be impossible to immediately forward park while entering because information is insufficient to determine the parking target area 600 for forward parking. Thus, there is a difference in that, after completely passing an empty parking area, if it is determined that the area to be parked is 600, the vehicle reverses again and performs forward parking.

8 is a flowchart of a parking assist method in case of reverse parking.

First, an image is obtained from the AVM system (S810), and a top view image 100 is generated by synthesizing the images using a plurality of cameras (S820). Based on the top view image 100, a parking division line 220 is calculated (S830), and a virtual parking reference line 450 is estimated and generated from the parking division line 220 (S840).

A parking target area 600 is determined based on the parking division line 220 and the virtual parking reference line 450 (S850), and the distance and deflection angle between the virtual parking reference line 450 and the vehicle are calculated (S860) to adjust the steering wheel. By controlling, a forward movement path is created so that the direction of the virtual parking reference line 450 and the driving direction of the vehicle are parallel (S870), and the steering wheel is controlled to move forward according to the created movement path (S875). At this time, the obstacle may be recognized and avoided using an ultrasonic sensor.

The parking division line 220 is calculated again using the new top-view image 100 generated according to the movement of the vehicle, and a more accurate parking target area 600 is determined by updating the predetermined parking target area 600 (S880). ). A reverse movement path is created to accurately park in the updated parking target area 600 (S890), and the steering wheel is controlled according to the created reverse movement path to reverse park (S895).

In the reverse movement path, if reverse parking is not possible at one time, reverse and forward are repeated and parked several times. Before parking several times, the number of repetitions according to the simulation result is displayed to the driver.

While the vehicle is moving forward or backward for parking, a moving path may be created by avoiding obstacles detected by the ultrasonic sensor.

During movement, the parking division line 220 is continuously recalculated, the virtual parking reference line 450 is re-estimated, the parking target area 600 is updated, and a more precise movement path is generated.

Above, the configuration of the present invention has been described in detail with reference to the accompanying drawings, but this is only an example, and those having ordinary knowledge in the technical field to which the present invention belongs can make various modifications and changes within the scope of the technical idea of the present invention. Of course this is possible. Therefore, the scope of protection of the present invention should not be limited to the above-described embodiments, but should be defined by the description of the claims below.

100: top view image
210: Parking reference line
220: Parking division line
300: feature point
410: reference boundary point
415: boundary point
420: Parking division line candidate group
440: parking compartment intersection
450: virtual parking reference line
S510: Searching for feature points
S521: Detecting Reference Boundary Points
S522: step of estimating the boundary line
S531: Step of extracting the first parking division line candidate
S532: Step of extracting a second candidate for the parking division line
S540: Calculating the parking division line
S550: step of predicting and updating information on parking division lines
S560: Calculating the intersection of parking division lines
S570: Step of estimating virtual parking reference line
S580: Step for determining parking target area
S600: Parking target area
S810: Acquiring a camera image
S820: AVM system creates a top view image
S830: Step of calculating parking division line
S840: Step of generating virtual parking reference line
S850: Step of determining a parking target area in a single image
S860: Calculating the distance and deflection angle between the virtual parking reference line and the vehicle
S870: Generating a forward movement path
S875: step forward by controlling the steering wheel
S880: Renewing the parking target area in the continuous image
S890: Creating a backward movement path
S895: step of reverse parking by controlling the steering wheel

Claims (12)

an AVM system unit that acquires an image around the vehicle using a camera and generates a top-view image using the acquired image;
an image processing unit generating an edge image from the top-view image;
a parking division line calculating unit calculating a parking division line from the top view image and the edge image;
a virtual parking reference line generation unit generating a virtual parking reference line from the parking division line; and
A parking target area determination unit determining a parking target area based on the parking division line and the virtual parking reference line.
including,
The parking division line calculation unit,
Searching for a plurality of feature points from the top-view image, extracting two boundary lines for each feature point, extracting a first parking division line candidate based on the two extracted boundary lines,
The average slope of the first parking division line candidates is calculated, the first parking division line candidate having a slope closest to the average slope is set as a parking candidate representative line, and the slope of the parking candidate representative line and the slope of the other parking division line first candidates Comparing the difference between and removing the noise for the first parking division line candidate,
Detecting a second parking division line candidate based on information on the line thickness, distance between adjacent lines, and line slope of the first parking division line candidates from which the noise has been removed, and the information of the second parking division line candidate satisfies the parking lot law A parking assist device that calculates a single image parking dividing line.
According to claim 1,
a movement path generation unit that calculates a distance and a deflection angle between the virtual parking reference line and the vehicle, and calculates a movement path in consideration of the parking target area; and
a steering wheel controller for moving the vehicle along the movement path;
Parking assistance device further comprising a.
According to claim 1,
Ultrasonic sensor unit for detecting cars and obstacles parked in the parking lot; and
A v2x sensor unit that communicates with the v2x device installed in the parked car and the parking facility to receive the location and vehicle information of the parked car and to receive information about the parking facility.
Including more,
Characterized in that the parking target area is determined using the vehicle and obstacle information recognized by the ultrasonic sensor unit and the car and parking lot facility information recognized by the v2x sensor unit
parking assist device.
According to claim 2,
Ultrasonic sensor unit for detecting cars and obstacles parked in the parking lot; and
The v2x sensor unit communicates with the v2x device installed in the car and the parking facility to receive the location and vehicle information of the car and to receive information about the parking facility.
Including more,
Characterized in that the movement route is generated in consideration of the vehicle and obstacle information recognized by the ultrasonic sensor unit and the vehicle and parking lot facility information recognized by the v2x sensor unit
parking assist device.
According to claim 1,
The parking division line calculator
From the top-view image, a plurality of feature points are searched using a top-hat filter in the driving direction of the vehicle, a boundary point on both sides of one of the feature points is found, the boundary point is rotated by 360 degrees, and two boundary lines are generated for each feature point. extract,
extracting a first parking division line candidate in which the difference between the slopes of the two extracted boundary lines is within a certain range;
Detecting the parking division line second candidate from the parking division line first candidates;
Characterized in that the parking division line is repeatedly calculated and updated from the continuous image
parking assist device.

According to claim 2,
The movement path includes a forward movement path and a backward movement path,
The forward movement path maintains a constant distance from the virtual parking reference line using the distance and the deflection angle, maintains a constant distance from an obstacle such as a parked vehicle, and sets the vehicle traveling direction to the direction of the virtual parking reference line. generated so that it remains parallel to ,
The rear movement path is generated so that the center line of the parking target area in which the final parking state is recognized coincides with the longitudinal axis of the vehicle,
Characterized in that the steering wheel controller controls the steering wheel according to the generated forward movement path and rear movement path.
parking assist device.
According to claim 1,
Further comprising a disabled parking compartment detection unit for detecting a disabled parking compartment mark,
The disabled parking section detection unit
If the driver does not set it as a disabled driving vehicle, it is judged as a non-parking area,
If the driver sets it as a disabled driving vehicle, parking is parked if the parking target area satisfies the conditions of the disabled parking area according to the law, otherwise it is judged as a non-parking area.
parking assist device.
According to claim 1,
Among the images obtained from the camera, external illumination is measured by analyzing the brightness of the exterior and surroundings of the vehicle,
Characterized in that the edge image is generated by adjusting the sensitivity according to the boundary point detection according to the measured external illuminance
parking assist device.
Obtaining an image captured using one or more cameras;
generating a top-view image by synthesizing the acquired images into a plan view;
converting the top-view image into an edge image;
Scanning the road surface in the vehicle traveling direction to detect light-colored feature points, recognizing two boundary lines for each feature point from boundary points around the detected feature points, and calculating parking division lines from the boundary lines;
calculating a parking section intersection point from the detected parking section line, and generating a virtual parking reference line by connecting the parking section intersection point; and
Determining a parking target area from the parking section line and the virtual parking reference line
including,
The step of calculating the parking division line,
Extracting a first parking division line candidate based on the two extracted boundary lines;
The average slope of the first parking division line candidates is calculated, the first parking division line candidate having a slope closest to the average slope is set as the parking candidate representative line, and the slope of the parking candidate representative line and the slope of the other parking division line first candidates Comparing the difference between and removing the noise for the first parking division line candidate,
Detecting a second parking division line candidate based on information on the line thickness, distance between adjacent lines, and line slope of the first parking division line candidates from which the noise has been removed, and the information of the second parking division line candidate satisfies the parking lot law A parking assistance method for calculating a single image parking segment.
According to claim 9,
setting a forward movement path such that a distance between the virtual parking reference line and the vehicle is kept constant, the driving direction of the vehicle is parallel to the direction of the virtual parking reference line, and a distance to an obstacle near a parking target area is kept constant;
setting a reverse movement path to reverse-park the traveling vehicle in the determined parking target area after moving forward;
Controlling a steering wheel to move along the forward movement path and the backward movement path.
More inclusive parking assistance methods.
According to claim 9,
The step of calculating the parking division line
(a) extracting a first parking division line candidate based on the distance between the pair of boundary lines and whether directions of the boundary lines match from the boundary line;
(b) extracting as a second parking division line candidate when the distance between adjacent first parking division line candidates and the direction of the first parking division line candidates have a value within a predetermined range; and
(c) Calculating as a parking division line when the line thickness, line direction, and distance between lines among the second parking division line candidates satisfy the law
Parking assistance method comprising a.
According to claim 9,
The generating of the top-view image is characterized by generating a plurality of top-view images according to the positional movement of the vehicle at finite time intervals,
The step of calculating the parking section is
Predicting the next parking division line in consideration of the parking division line calculated from the top view image and the position and moving speed of the vehicle at the time of calculation,
The parking division line is updated by comparing the parking division line calculated from the continuous top-view image of the next period with the parking division line predicted in the previous period and correcting the prediction parameter,
Saving the calculation result and prediction result of the parking division line for each period
Parking assistance method characterized by.

Images