JP5633376B2 - Parking assistance system - Google Patents

Description

  The present invention relates to a parking assistance system that assists a driving operation for parking when a driver of a vehicle parks the vehicle in a desired area.

  As a conventional parking assistance system, for example, the one shown in Patent Document 1 is known. In the system of Patent Literature 1, when an arbitrary position is designated by a user in a bird's-eye view image that displays a situation around a vehicle, the designated position is set as a parking position, and the periphery of the designated position is imaged. The direction detected by analysis is set as the parking direction. And a parking frame is set based on the set parking position, parking direction, and the size of the own vehicle, and a parking frame figure representing the parking frame is displayed superimposed on the bird's-eye view image. In this way, the user can set a target parking frame by designating an arbitrary position while viewing the bird's-eye view image.

  When the parking frame is set, the entire course from the current position of the host vehicle to the parking position is calculated based on the position and direction of the parking frame, and is displayed superimposed on the bird's-eye view image.

Japanese Patent Application Laid-Open No. 2005-239048

  According to the parking support system disclosed in Patent Document 1, since the parking position can be specified in the bird's-eye view image including the host vehicle and the surroundings of the vehicle displayed, the positional relationship between the host vehicle and the surrounding environment It is easy to understand intuitively, and a parking frame can be easily set at a desired position.

  However, in the parking assistance system described above, for parking lots that are used frequently and daily, for example, at home parking lots, the parking position must be specified in the bird's-eye view image every time parking is performed. Some may feel annoying.

  The present invention has been made in view of the above-described points, and it is an object of the present invention to provide a parking support system capable of reducing the effort for setting a parking frame particularly for a parking lot used daily. To do.

In order to achieve the above object, a parking assistance system according to claim 1 is provided.
Marking patterns provided on the road surface to determine the positions of at least two intersecting parking frame lines as parking frame lines indicating areas where the vehicle should be parked,
A photographing means mounted on the vehicle for photographing an image around the vehicle;
In the image photographed by the photographing means, a landmark pattern detection means for searching for a landmark pattern and detecting the landmark pattern;
Based on the position of the mark pattern detected by the mark pattern detecting means, the positions of two intersecting parking frame lines are determined, and the positional relationship between the positions of these parking frame lines and the vehicle is notified to the vehicle driver. An informing means ,
The mark pattern is placed at a corner portion of a region where the vehicle is to be parked, and indicates a corner position and a position of two parking frame lines extending from the corner position .

Thus, the parking assist system of claim 1, utilizing a mark pattern formed on the road surface, determining the position of the parking frame line. In particular, the mark pattern is placed at the corner of the area where the vehicle is to be parked, and indicates the corner position and the positions of the two parking frame lines extending from the corner position. The position of two intersecting parking frame lines surrounding the area to be parked can be determined. Therefore, once a mark pattern is provided at a desired position in a frequently used parking lot or the like, there is no need to perform a special operation such as setting a parking position on the screen every time parking is performed.

  Since the mark pattern determines the positions of at least two intersecting parking frame lines, parking assistance can be performed so that the vehicle can be parked at an appropriate position on the basis of the two parking frame lines. . And the parking assistance system can uniquely determine the position of the parking frame line indicating the area where the vehicle is to be parked by determining the positions of the two parking frame lines by the mark pattern.

  Preferably, the mark pattern is drawn on a movable sheet placed on a road surface. In this way, if the mark pattern is drawn on the seat, the mark pattern can be easily provided on a desired road surface position simply by placing the seat in a frequently used parking lot. Furthermore, if a sheet on which a mark pattern is drawn is carried, it is not used on a daily basis, but it is possible to receive parking assistance by a parking assistance system even for a parking lot with a high degree of parking difficulty. In other words, if the vehicle driver needs to park in a parking lot with a high degree of parking difficulty such as a narrow parking lot, a parking lot with many obstacles, or a parking lot with the possibility of derailment, the target of the vehicle driver is A portable seat can be temporarily installed at a position corresponding to the parking position. Thereby, it becomes possible to park safely even in a parking lot where parking difficulty is high.

Sheet according to claim 2, when placed respectively in the two corners adjacent areas should be parked vehicle, as claimed in claim 3, the notification means, inter corners sheet is placed With respect to the parking frame line, it is preferable to determine the position of the parking frame line so as to connect the corner positions indicated by the mark patterns of the respective sheets. When the positions of two parking frame lines that intersect each sheet are determined based on the mark pattern drawn on each sheet, the parking frame lines overlap between the corners where each sheet is placed. Will end up. For such a problem, as described above, by unifying the parking frame line by unifying the parking frame line so as to connect the corner positions indicated by the mark patterns of the respective sheets, it is unified. A single parking frame line can be defined.

According to a fourth aspect of the present invention , it is preferable that the notification means issues a warning to the driver of the vehicle when the distance between the parking frame line and the vehicle falls below a predetermined threshold. Thereby, the driver | operator of a vehicle can recognize correctly that the vehicle approached below the predetermined threshold distance with respect to the parking frame line with a warning.

According to a fifth aspect of the present invention, the notifying means has a display device provided in the passenger compartment, displays a vehicle peripheral image based on the image photographed by the photographing means on the display device, and positions of the mark pattern It is preferable to display the virtual parking frame line superimposed on the vehicle peripheral image at the position of the parking frame line determined based on the above. Thereby, the driver | operator of a vehicle can grasp | ascertain the positional relationship of a vehicle and a parking frame line from the vehicle periphery image displayed on a display apparatus.

According to a sixth aspect of the present invention, the notifying means may obtain the shortest distance from the vehicle to the virtual parking frame line and display the obtained shortest distance on the vehicle peripheral image. Thereby, the driver of the vehicle can grasp the positional relationship between the vehicle and the parking frame line more objectively and clearly.

As described in claim 7 , the notification means may superimpose and display a virtual corner pole at a position corresponding to the intersection of two intersecting parking frame lines in the vehicle periphery image. This makes it easier for the driver of the vehicle to recognize the positional relationship with the corner where the two parking frame lines intersect from the vehicle peripheral image displayed on the display device.

As described in claim 8 , the notification means may change the display form of the corner pole according to the degree of approach between the vehicle and the corner pole. For example, as the display form of the corner pole, the display color is changed from blue → green → yellow → red as the vehicle gradually approaches from a state where the vehicle exists at a long distance. As a result, the driver of the vehicle can intuitively understand the positional relationship between the vehicle and the corner pole.

According to a ninth aspect of the present invention, the notifying means has a display device provided in the vehicle interior, determines a rectangular parking area based on at least the positions of two parking frame lines that intersect, and this parking area calculating a route that the vehicle follows up fit in, as the positional relationship between the position and the vehicle parking frame lines, the path of the vehicle to fit in the parking area and parked area displayed on the Viewing device, the driver of the vehicle On the basis of the instruction, a parking control unit that performs parking control of the vehicle may be provided so that the vehicle automatically moves according to the route displayed by the notification unit.

When the positions of at least two intersecting frame lines are determined from the mark pattern, a rectangular parking area for parking the vehicle can be obtained by using the longitudinal and lateral lengths of the vehicle. Can be determined. By displaying these parking areas and routes as the positional relationship between the position of the parking frame and the vehicle, the driver can confirm that the parking area of the vehicle is set as intended and the route to the parking area. , You can check if there is any safety problem. Furthermore, in the parking support system according to claim 9 , the parking control means automatically moves the vehicle from the current position to a position that fits in the parking area according to the displayed route based on the driver's instruction. Perform parking control. Therefore, it is not necessary for the driver to manually set a parking area for parking the vehicle, and the labor for executing the parking control can be greatly reduced.

The mark pattern is arranged at a corner portion of the rectangular parking area, and a plurality of mark patterns located at both ends of each side of the parking area are searched by the position specifying means. When this is done, the notification means may calculate a route passing on the side between the searched landmark patterns. When there is an obstacle that hinders the vehicle from traveling on the route passing between the mark patterns, the mark pattern may be hidden by the obstacle in the image taken by the photographing means mounted on the vehicle. Many. Therefore, when a plurality of landmark patterns located at both ends of each side of the parking area are projected on the image photographed by the photographing means, an obstacle that hinders the progress of the vehicle between the plurality of landmark patterns. It can be said that there is a low possibility of existence. Therefore, by calculating a route that passes between the searched landmark patterns, it is possible to present a route that is likely to allow the vehicle to travel safely.

As described in claim 11 , the notifying means calculates and displays a plurality of different routes as a route that the vehicle follows until it fits in the parking area, and the parking control means can select from the displayed plurality of routes. When one route is selected by the driver, parking control may be performed according to the selected route. For example, if there is an obstacle (another vehicle, a pillar, a wall, etc.) adjacent to the parking area of the vehicle, the vehicle may come into contact with the obstacle depending on the route to the parking area. In this regard, as described above, if a plurality of routes are calculated and displayed, the possibility of presenting a route that can safely park the vehicle can be increased even if there are obstacles. The use opportunity of parking control by a person increases, and more effective parking assistance can be performed.

According to a twelfth aspect of the present invention, it is preferable that the route that the vehicle follows until it falls within the parking area is superimposed and displayed on the vehicle periphery image based on the image captured by the image capturing means. As a result, it is possible to check obstacles that actually exist around the vehicle, and to check the positional relationship between the parking area and the route, and the obstacles from the image displayed on the display device.

According to a thirteenth aspect of the present invention, the notification unit includes a bird's-eye image generation unit that generates a bird's-eye view image obtained by performing bird's-eye conversion on an image captured by the image capturing unit. The bird's-eye image generated by the bird's-eye image generation unit may be displayed side by side on the display device. Alternatively, as described in claim 14 , the notifying unit outputs, as the vehicle peripheral image, an image photographed by the photographing unit and a bird's-eye view image generated by the bird's-eye image generating unit in accordance with an instruction from the driver of the vehicle. May be switched and displayed on the display device. In any case, the driver of the vehicle can grasp the positional relationship between the vehicle and the parking frame from the vehicle peripheral image displayed on the display device.

It is a block diagram which shows the structure of the parking assistance system by 1st Embodiment of this invention. It is a figure which shows an example of the parking assistance sheet | seat used in this parking assistance system. It is a flowchart which shows the process for parking assistance performed in image processing ECU of the parking assistance system of FIG. It is a figure which shows the example of the point with a SIFT feature-value in a parking assistance sheet | seat. It is a figure which shows the correspondence of the point with the basic SIFT feature-value preserve | saved previously regarding the parking assistance sheet | seat, and the point with the test SIFT feature-value extracted in the real image. It is a figure which shows an example of the real image with a guide displayed on a display apparatus as a vehicle periphery image. It is a figure which shows an example of the bird's-eye view image with a guide displayed on a display apparatus as a vehicle periphery image. It is a block diagram which shows the structure of the parking assistance system by 2nd Embodiment. It is explanatory drawing for demonstrating the setting process of a parking area | region in case only the three parking assistance sheets 40 are detected by the parking assistance sheet search part 21 of image processing ECU20. It is explanatory drawing for demonstrating the process which calculates the path | route for moving a vehicle to the parking position in a parking area in case the three parking assistance sheets 40 are detected. It is explanatory drawing for demonstrating the setting process of a parking area | region in case only the two parking assistance sheets 40 located in a short direction of the parking area | region are detected. It is explanatory drawing for demonstrating the process which calculates the path | route for moving a vehicle to the parking position in a parking area, when the two parking assistance sheets 40 are detected. It is a figure which shows the intermediate position of a vehicle utilized when setting the path | route shown in FIG. It is explanatory drawing for demonstrating the setting process of the parking area | region in case only the one parking assistance sheet | seat 40 is detected. It is a figure which shows the example of a display which calculates and displays a several different path | route as a path | route which a vehicle follows until it reaches the parking position in a parking area.

(First embodiment)
Hereinafter, a parking assistance system according to an embodiment of the present invention will be described in detail based on the drawings. FIG. 1 is a configuration diagram showing the overall configuration of the parking assistance system according to the first embodiment of the present invention. In the present embodiment, an example will be described in which a CCD camera 10 as a photographing unit is mounted on the rear part of the vehicle to photograph the state behind the vehicle.

  As shown in FIG. 1, the parking support system 100 includes a CCD camera 10, an image processing ECU 20, a display device 30, and a parking support sheet 40.

  The CCD camera 10 is installed at the rear part of the vehicle, specifically, at the rear door or trunk of the vehicle so that the optical axis is downward from the direction parallel to the ground and the photographing area includes the ground near the rear of the vehicle. Is done. The imaging range behind the vehicle of the CCD camera 10 is determined in advance. When the CCD camera 1 is installed at the rear of the vehicle, the CCD camera 1 is set so that the actual imaging range matches the predetermined imaging range. The mounting direction is adjusted (calibration).

  The image processing ECU 20 includes a microcomputer having a CPU, a ROM, a RAM, an I / O, a bus line for connecting them, and the like. However, in FIG. 1, functions executed by the image processing ECU 20 are shown as a functional block diagram. As illustrated in FIG. 1, the image processing ECU 20 includes a parking assistance sheet search unit 21, a contact distance calculation unit 22, and an image generation unit 23 as functional blocks.

  The parking assistance sheet search unit 21 searches and detects the parking assistance sheet by image processing when the parking assistance sheet 40 is displayed in the camera image (actual image) taken by the CCD camera 10.

  The parking assist sheet 40 is placed at the corner of the area where the vehicle is to be parked (parking area), and the pattern drawn on the surface of the parking area has a corner position of the parking area and two parking frame lines extending from the corner position. It shows the position. The parking assist sheet 40 is made of a material such as rubber, metal, plus chip, and can be carried by a vehicle occupant. As shown in FIG. 2, the parking assistance sheet 40 has a right-angled pattern so that the direction of the corner can be indicated. Then, the first graphic pattern 41 having a right-angle pattern is formed at the corner position so that the right-angle pattern can be easily identified in the image processing, and the second pattern along the two orthogonal straight lines extending from the corner. And third graphic patterns 42 and 43 are formed.

  The first to third graphic patterns 41 to 43 are formed in different designs. Therefore, the position and direction of the parking assistance sheet 40 can be determined using these first to third graphic patterns 41 to 43 as clues. Specifically, by recognizing each of the first to third graphic patterns, the position and orientation of the right-angled pattern represented on the parking assist seat 40 are determined from their mutual positional relationship. As a result, the position of the corner and the positions of two orthogonal straight lines extending from the corner can be specified. The corner position thus identified corresponds to one vertex of the rectangular parking area, and the two straight lines extending from the corner are the first and second parking frames as shown in FIG. Corresponds to a line.

  In addition, the parking assistance sheet 40 is formed so that light is diffusely reflected on its surface so that a high detection rate can be obtained by the image recognition technology, and the first to third graphic patterns 41 to 43 are formed. The pattern of the parking assistance sheet 40 including the image is created using a paint that clearly shows a difference in brightness contrast. The entire pattern drawn on the parking support sheet 40 corresponds to the mark pattern in the present invention.

  The contact distance calculation unit 22 calculates the minimum distance between the corner position and the two parking frame lines specified by the parking assistance sheet search unit 21 and the host vehicle as the contact distance. Specifically, the minimum distance between each point on the image obtained by orthogonally projecting the host vehicle on the road surface, the corner position, and the two parking frame lines is calculated, and this is set as the contact distance.

  The CCD camera 10 has a mounting position determined for each vehicle mounted. Therefore, an image obtained by orthogonally projecting the own vehicle can be calculated and stored in advance in the XY coordinate system corresponding to the road surface with the mounting position of the camera as a reference. When the corner position and the positions of the two parking frame lines extending from the corner are specified from the search result of the parking assist sheet 40 in the camera image photographed by the CCD camera 10, the photographing area of the CCD camera 10 is determined in advance. Since the parking support sheet 40 is placed on the road surface, the corner position and the positions of the two parking frame lines can be determined in the XY coordinate system corresponding to the road surface. Then, in the XY coordinate system, it is possible to calculate the minimum distance between the image of the host vehicle whose position is determined, the corner position, and the two parking frame lines.

  The image generation unit 23 includes a camera image captured by the CCD camera 10, a corner position determined by the parking support sheet search unit 21 and the positions of the two parking frame lines, and a contact calculated by the contact distance calculation unit 22. Based on the distance, a real image with guide and / or a bird's-eye image with guide are generated and output to the display device 30.

  As shown in FIG. 6, the real image with guide is a virtual parking pole for indicating a corner position and a virtual parking frame for indicating the positions of two parking frame lines in the camera image taken by the CCD camera 10. A line and a distance display for indicating the minimum distance between each virtual parking frame line and the host vehicle are superimposed. These superimposed corner corners, virtual parking frame lines, and distance displays are drawn by computer graphics. FIG. 6 shows an example of a partial image including a corner portion where the parking support sheet 40 is placed, which is a part of the actual image with guide.

  By superimposing and displaying the virtual parking frame line, the driver of the vehicle can clearly grasp the positional relationship between the vehicle and the parking frame line from the real image with guide displayed on the display device 30. Further, by displaying the shortest distance from the parking frame line, the vehicle driver can more objectively and clearly grasp the positional relationship between the vehicle and the parking frame line.

  Furthermore, by superimposing and displaying the virtual corner pole, the vehicle driver can easily recognize the positional relationship between the host vehicle and the corner position from the actual image with guide displayed on the display device 30.

  The virtual corner pole is colored, and the display color changes according to the shortest distance between the virtual corner pole and the host vehicle. For example, when the shortest distance is longer than 90 cm, it is displayed in blue, when it is 60 cm to 90 cm, it is displayed with green, when it is 30 cm to 60 cm, it is displayed in yellow, and when it is 30 cm or less, it is displayed in red. Thus, by changing the display color according to the shortest distance, the driver of the vehicle can intuitively understand the degree of approach between the host vehicle and the virtual corner pole. In addition to changing the display color, the transparency of the virtual corner pole (displayed darker as it approaches) or brightness (displayed brighter as it approaches) may be changed.

  Further, as shown in FIG. 7, the bird's-eye image with guide is an image showing the own vehicle, a virtual parking frame line, and a minimum distance between each virtual parking frame line and the own vehicle in the bird's-eye image obtained by bird's-eye conversion of the camera image. Etc. are superimposed. By displaying such a bird's-eye view image with a guide, the driver of the vehicle can intuitively grasp the posture of the host vehicle in the parking lot and the distance to the parking frame line. In addition, although the superimposed own vehicle is the entire vehicle in the example shown in FIG. 7, it may be only the rear portion of the vehicle. Further, a virtual corner pole may be superimposed and displayed on the bird's-eye image with guide.

  Only one or both of the real image with guide and the bird's-eye image with guide may be generated by the image generation unit 23. When the image generation unit 23 generates both the guided real image and the guided bird's-eye image, the guided real image and the guided bird's-eye image may be displayed side by side on the display device 30 or the vehicle. A selection switch operated by the driver may be provided, and only one of the images selected by the driver of the vehicle may be switched and displayed.

  The display device 30 is attached to the instrument panel of the vehicle so that it can be viewed by the driver. On the display device 30, the real image with guide and / or the bird's-eye image with guide generated by the image generation unit 23 is displayed.

  In addition to the above-described configuration, for example, it is determined whether or not the above-described contact distance is equal to or less than a predetermined threshold value. When the contact distance is equal to or less than the predetermined threshold value, a warning sound or sound is output or the display device 30 A warning unit that issues a warning to the driver of the vehicle by displaying a warning message may be provided in the image processing ECU 20.

  Next, a control process for parking assistance executed by the image processing ECU 20 in the parking assistance system according to the present embodiment will be described based on the flowchart of FIG. Note that the control process shown in the flowchart of FIG. 3 is repeatedly executed from when the vehicle starts to park until it stops.

  Here, in the image processing ECU 20, various methods can be adopted as a method for recognizing the pattern (marking pattern) drawn on the parking assistance sheet 40 by image processing. For example, images of the parking support sheet 40 previously taken from various shooting angles are prepared as template images, and the images are cut out from the camera images taken by the CCD camera 10 with a cutout window having a predetermined size. Then, the pattern of the parking support sheet 40 may be recognized by comparing with the template image (so-called template matching). In this case, it is possible to cope with a change in the distance to the parking assist seat 40 by changing the size of the cutout window.

  However, in this embodiment, the pattern of the parking assistance sheet 40 is recognized using a SIFT (Scale Invariant Feature Transform) feature amount. This is because the SIFT feature amount is a feature amount that is invariant to the scale and rotation of the image including the recognition target, and robust recognition can be performed even for a minute affine transformation. Since the SIFT feature value is described in detail in Reference Document 1 below, the description of the SIFT feature value will be simplified in the following description of the control process. Further, a feature value called ASIFT that is invariant to affine transformation has been reported (Reference Document 2). In the present embodiment, an example using a normal SIFT feature value will be described.

(Reference 1)
DG Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004), pp. 91-110.
(Reference 2)
JM Morel and G. Yu, "ASIFT, A New Framework for Fully Affine Invariant Image Comparison," SIAM Journal on Imaging Sciences, vol. 2, issue 2, 2009.
First, a general SIFT feature value calculation method will be described. The SIFT feature value calculation is composed of two stages: detection of feature points suitable for feature extraction and description of feature values that are invariant to scale change, rotation, and the like.

  In detection of feature points, first, extreme values that are candidate feature points are detected. Therefore, the image is smoothed using Gaussian filters having different scales. Next, a DoG (Difference of Gaussian) image which is a difference image of each smoothed image is obtained. A plurality of DoG images are obtained by performing the process for obtaining the DoG images between different scales which are increased by k times.

  Next, extreme values are detected from the obtained DoG image, and feature points and scales are determined. The extreme value is detected by taking three DoG images with different scales as one set, and comparing the DoG value of the target pixel with 26 pixels in the vicinity of the target pixel including the DoG images of the upper and lower scales. That is, when the DoG value of the target pixel becomes maximum, the target pixel is set as a feature point candidate. Further, candidate points having DoG values, main curvatures, and DoG values of subpixels within a certain range are set as feature points. Scale information is assigned to the feature points.

  Next, regarding the description of the feature amount, first, a direction that is representative of each detected feature point is obtained. Specifically, for each feature point, the direction histogram of the luminance gradient is calculated in the vicinity region, and the direction with the highest frequency is searched. Then, a local coordinate system is created by rotating the image coordinate system in the direction around the feature point.

  In this local coordinate system, a local area having a size proportional to the scale information held around the feature point is considered, and the area is divided into 4 × 4 blocks. Then, a histogram of luminance gradients in eight directions is created for each block. Thus, 4 × 4 × 8 = 128-dimensional SIFT feature value is obtained.

  In the present embodiment, the SIFT feature value calculation process described above is applied in advance to the image obtained by capturing the parking support sheet 40 shown in FIG. Is calculated and stored in the image processing ECU 20. The SIFT feature value of each feature point is referred to as a basic SIFT feature value. FIG. 4 shows an example of feature points for which basic SIFT feature values have been calculated for the pattern of the parking assistance sheet.

  In the control process shown in the flowchart of FIG. 3, first, in step S110, a camera image taken by the CCD camera 10 is captured. In subsequent step S120, a rectangular detection window having a predetermined size is set in the camera image, and an image in the detection window is cut out. The position of the detection window is updated so as to be moved by a predetermined pixel (for example, 3 pixels) in the camera image. Then, the movement of the detection window is repeated so that the movement by the detection window covers the entire image area. The size of the window is set so that one parking assistance sheet 40 is large enough to enter, and two parking assistance sheets 40 separated in the vehicle width direction do not enter at the same time. Is done.

  In step S130, it is determined whether or not the detection window has been moved to cover the entire image area, that is, the last detection window, and the detection window could not be updated in step S120. At this time, if it is determined as the last detection window, the process proceeds to step S210. On the other hand, if an image is cut out by a new detection window instead of the last detection window, the process proceeds to step S140.

  In step S140, a SIFT feature value is calculated for each feature point of the image by applying the SIFT feature value calculation process described above to the image cut out by the detection window. This calculated SIFT feature value is referred to as a test SIFT feature value. In subsequent step S150, each feature point for which the basic SIFT feature value has been calculated is associated with each feature point for which the test SIFT feature value has been calculated. Specifically, one test SIFT feature amount having the closest Euclidean distance is selected with respect to the basic SIFT feature amount calculated for each feature point. At this time, if the Euclidean distance is smaller than a predetermined threshold value, the feature point having the test SIFT feature value is set as a corresponding point, and a large point is assumed to have no corresponding point. By performing such processing for the basic SIFT feature values of all feature points, the total number of corresponding points is calculated.

  In step S160, it is determined whether the total number of corresponding points is greater than a predetermined threshold. At this time, if the total number of corresponding points is larger than a predetermined threshold value, it is considered that a parking assistance seat candidate is detected in the image cut out by the detection window. When a parking assistance sheet candidate is detected, the process proceeds to step S170, and the three-dimensional position of the parking assistance sheet is determined from the image coordinates of the corresponding point in the camera image according to the following procedure.

  Let (x, y) be the image coordinates of the feature points of the corresponding camera image. The image coordinates (x, y) have the relationship shown in the following formula 1 with the focal length f of the CCD camera 10, the three-dimensional coordinates (X, Y, Z) with reference to the road surface.

(Equation 1)
x = f · X / Z
y = f · Y / Z
In addition, since the mounting position and mounting angle of the CCD camera 10 with respect to the vehicle are determined in advance, the formula of the plane satisfied by the points on the road surface can be expressed by the following formula 2.

(Equation 2)
aX + bY + Z = C
Here, since the parking assistance sheet 40 is placed on the road surface, X, Y, and Z can be uniquely determined from the equations (1) and (2), and the positions of corresponding points in the three-dimensional coordinates. Can be requested. The calculation of this position is performed for all the points that can be dealt with.

  In step S180, it is determined whether or not the arrangement of the corresponding points in the three-dimensional coordinates obtained in step S170 matches the basic point arrangement corresponding to the pattern of the parking assistance sheet 40. That is, it is determined whether or not the corresponding point arrangement and the basic point arrangement form a similar shape. In the process of step S180, when it is determined that the arrangement of the corresponding points and the basic point arrangement are similar, it can be considered that the parking support sheet 40 is detected in the image cut out by the detection window. Proceed to On the other hand, if it is determined that the similar shape is not formed, it is determined that the parking support sheet candidate does not correspond to the parking support sheet, and the process returns to step S120.

  In step S190, the position of the corner and the positions of the two parking frame lines extending from the corner are obtained. That is, the position of the corner and the position of the parking frame line are defined in advance by the coordinates of feature points having basic SIFT feature values. Therefore, when a parking assistance sheet is detected and the point arrangement on the sheet is obtained, the position of the corner and the position of the parking frame line are obtained. In step S200, the minimum distance between the calculated corner position and the two parking frame lines and the host vehicle is calculated as the contact distance.

  In step S210, which is executed when the detection window has been moved so as to cover the entire image area, the corner position and parking frame line position calculated in step S190, and the contact distance calculated in step S200. Based on the above, the above-described real image with guide and / or bird's-eye view image with guide are displayed as the vehicle peripheral image. Thereby, the driver | operator of a vehicle can grasp | ascertain correctly the positional relationship with the parking frame line and corner which show the area | region which should park. Therefore, it is possible to perform an operation for parking the vehicle safely and reliably.

  In this embodiment, since the parking support sheet 40 that can indicate the position of the parking frame line and the corner point is used, once the parking support sheet 40 is installed in a frequently used parking lot or the like, There is no need to perform a special operation such as setting a parking position on the screen every time the vehicle is parked.

  Since the pattern of the parking assistance sheet 40 determines the position of a corner point and two intersecting parking frame lines extending from the corner point, the vehicle is brought to an appropriate position on the basis of the two parking frame lines. Parking assistance can be performed so that parking is possible. Furthermore, in this embodiment, since the parking assistance sheet | seat 40 mentioned above is used, the parking assistance system 100 can define uniquely the position of the parking frame line which shows the area | region which should park a vehicle.

  In addition, since the parking assistance seat 40 is portable for the vehicle occupant, the parking assistance seat 40 is not used on a daily basis, but the parking assistance system 100 can receive parking assistance even for a parking lot with a high degree of parking difficulty. It becomes possible. In other words, if the vehicle driver needs to park in a parking lot with a high degree of parking difficulty such as a narrow parking lot, a parking lot with many obstacles, or a parking lot with the possibility of derailment, the target of the vehicle driver is The portable parking support sheet 40 can be temporarily installed at a position corresponding to the parking position. Thereby, it becomes possible to park safely even in a parking lot where parking difficulty is high.

  Although the first embodiment of the present invention has been described above, the parking support system 100 according to the first embodiment can be variously modified without being limited to the specific examples described above.

  For example, in the first embodiment described above, an example in which one parking assistance sheet 40 is arranged at a corner portion of a parking area has been described. However, a plurality of parking assistance sheets 40 may be used in combination. For example, if two parking assistance sheets 40 are installed at the diagonal corner positions of the target parking area, all four sides of the parking frame line can be shown. Moreover, if the two parking assistance sheets 40 are installed at two corner positions on both ends of one side where the target parking area is located, three sides of the parking frame line can be shown. However, in this case, if the positions of the two parking frame lines intersecting each seat 40 are determined based on the pattern drawn on each parking assist seat 40, the corner where each seat 40 is placed. In the meantime, the parking frame lines will overlap. In order to prevent such a problem, the position of the parking frame line may be determined so as to connect the corner positions determined by the pattern of each sheet 40. Thereby, the overlapping of a parking frame line can be eliminated and the unified one parking frame line can be defined.

  Further, in the above-described embodiment, the position of the corner point and the two parking frame lines extending from the corner are determined by one parking assistance sheet 40. However, as the parking support sheet, only a pattern for determining the position of the parking frame line is drawn (for example, only the second or third graphic pattern of the parking support sheet 40 described above is drawn). The positions of two intersecting parking frame lines may be determined by combining them. In this case, the corner position is defined by the intersection of two parking frame lines.

  In the first embodiment described above, the example in which the CCD camera 10 is attached to the rear part of the vehicle has been described. However, the camera may be attached to the front part of the vehicle and photograph the situation in front of the vehicle. It may be attached to the side portion of the vehicle and photograph a side view. Furthermore, a plurality of cameras may be mounted on the vehicle.

  Furthermore, the seat 40 may be disposed at a position indicating the parking frame line on the near side instead of the parking frame line on the far side of the parking area of the vehicle. In this case, for example, in addition to the distance display with the parking frame line grasped from the pattern of the seat 40, the position of the remaining (back side) parking frame line is determined in consideration of the vertical direction and the horizontal direction of the vehicle. You may perform distance display etc. with the defined parking frame line.

(Second Embodiment)
Next, a parking assistance system according to a second embodiment of the present invention will be described with reference to the drawings. In addition, about the structure similar to 1st Embodiment mentioned above, detailed description is abbreviate | omitted by providing the same reference number.

  In the first embodiment described above, a real image with guide and / or a guide in which a virtual corner pole, a virtual parking frame line, a minimum distance from each virtual parking frame line, and the like are superimposed and displayed on an image taken by a camera. The attached bird's-eye view image was displayed.

  On the other hand, the parking support system according to the present embodiment determines a rectangular parking area based on the positions of at least two intersecting parking frame lines, and calculates a route that the vehicle follows until it fits in the parking area. . Then, as the information indicating the positional relationship between the position of the parking frame line and the vehicle, the parking area and the route that the vehicle follows until it falls within the parking area are displayed. By displaying the parking area and route in this way, the driver confirms whether the parking area of the vehicle is set as intended and whether there is a safety problem on the route to the parking area. Can do.

  Furthermore, the parking support system according to the present embodiment automatically moves the vehicle within the parking area when the driver of the vehicle is instructed to execute parking control that automatically moves the vehicle to the parking area according to the displayed route. The parking control to move to is executed. Accordingly, since it is not necessary for the driver to manually set a parking area for parking the vehicle, it is possible to greatly reduce the trouble of executing the parking control.

  FIG. 8 is a configuration diagram showing the configuration of the parking support system according to the present embodiment. As shown in FIG. 8, the image processing ECU 20 includes a parking area and route calculation unit 24 instead of the contact distance calculation unit 22.

  The parking area and route calculation unit 24 parks the vehicle using the corner position determined by the parking support sheet 40 searched by the parking support sheet search unit 21 and two straight lines (parking frame lines) extending from the corner position. A rectangular parking area is defined for this purpose. Furthermore, when the vehicle is moved from the current position of the vehicle to the parking position in the parking area, a route that the vehicle follows is calculated.

  The parking area setting process by the parking area and route calculation unit 24 will be described below. First, as shown in FIG. 9, the parking support sheet 40 is placed at the corners of the four corners of the rectangular parking area. Of these, only three parking support sheets 40 are parked by the image processing ECU 20. The case where it is detected by the support sheet search unit 21 will be described.

  In this case, as for the parking frame lines on the two sides of the parking area sandwiched by the three detected parking assistance sheets 40, the patterns of the respective parking assistance sheets 40 are the same as described in the first embodiment. The position of the parking frame line is determined so as to connect the corner positions determined by. And the position of the remaining two parking frame lines is determined from a straight line determined by the detected pattern of the parking assistance sheet 40. In this way, when the positions of the remaining two parking frame lines are determined, the remaining one corner position can also be determined from the intersection position.

  When three parking assistance sheets 40 are detected, a rectangular parking area surrounded by the parking frame lines on the four sides can be determined in this way. The parking area and route calculation unit 24 stores the vertical and horizontal lengths of the host vehicle, and the determined parking area is suitable for the vertical and horizontal lengths of the host vehicle. It may be determined whether or not it is insufficient, and if it is determined that it is insufficient, a warning to that effect may be made.

  Next, as shown in FIG. 11, a description will be given of a parking area setting process in the case where only two parking assistance sheets 40 arranged in the short direction of the parking area are detected due to an obstacle or the like. In this case, the positions of the parking frame lines on one side of the parking area sandwiched between the two parking support sheets 40 are determined so as to connect the corner positions determined by the respective parking support sheets 40. And about the parking frame line of two sides extended at right angles with respect to the parking frame line, the position is defined from the straight line defined by the pattern of each parking assistance sheet 40. Furthermore, the parking frame line on the other side is determined so that the vehicle can be accommodated using the longitudinal length of the vehicle stored in the parking area and the route calculation unit 24. In this case, it is determined whether the parking area surrounded by the parking frame line is appropriate or insufficient for the lateral length of the vehicle, and if it is insufficient, a warning to that effect is given. You may make it do.

  Furthermore, as shown in FIG. 14, the setting process of the parking area when only one parking assistance sheet 40 is detected will be described. In this case, one corner position of the parking area and the positions of the two parking frame lines extending from the corner position can be determined from the pattern of the single parking assistance sheet 40. Accordingly, based on the two parking frame lines, the remaining two sides of the parking frame lines that divide the parking area in which the host vehicle can be accommodated can be determined. That is, the positions of the remaining two sides of the parking frame line at a position perpendicular to the parking frame line determined from the parking assistance sheet 40 and adding a predetermined margin distance to the longitudinal and lateral lengths of the host vehicle. Can be determined.

  As described above, the parking area and route calculation unit 24 can set appropriate parking areas in accordance with the number of parking assistance sheets 40 detected by the parking assistance sheet search unit 21. In addition, when all the four parking assistance sheets 40 are detected, the position of the parking frame line can be determined so that the corner positions determined by the parking assistance sheets 40 are connected to each other.

  Next, route calculation processing by the parking area and route calculation unit 24 will be described. When the parking area and route calculation unit 24 determines the parking area based on the detected parking assistance sheet 40, the parking area and route calculation unit 24 calculates the route that the vehicle follows from the current position of the vehicle to the parking position in the parking area. Since the algorithm for calculating the route is known, the description thereof is omitted.

  However, in the present embodiment, when a plurality of parking support sheets 40 positioned at both ends of each side of the parking area are detected, the parking area and route calculation unit 24 is between the detected parking support sheets 40. The route passing on the parking frame line is calculated. When there is an obstacle that hinders the vehicle from traveling on the path that passes between the parking assistance sheets 40, the parking assistance sheet 40 is caused by the obstacle in the image taken by the CCD camera 10 mounted on the vehicle. Often hidden. Therefore, in a reverse view, when a plurality of parking support sheets 40 located at both ends of each side of the parking area are projected on the image photographed by the CCD camera 10, the space between the plurality of parking support sheets 40 is shown. In addition, it can be said that there is a low possibility that there is an obstacle that hinders the progress of the vehicle. Therefore, a route that is likely to be able to travel safely can be presented by calculating a route that passes between the plurality of searched parking assistance sheets and displaying the route.

  Specifically, for example, as shown in FIG. 10, when parking support sheets 40 arranged in three corners of four corners of the parking area are detected, these three parking assists are detected. A route passing through the two parking frame lines between the seats 40 is calculated.

  In this case, first, a route for moving the vehicle from the current position of the vehicle to the parking position in the parking area is calculated, and the calculated route is the two parking frames between the three parking support sheets 40. If it passes on the line, the route may be adopted as it is, and if it does not pass, the route for moving the vehicle to the parking position may be calculated by repeating calculation of different routes until it passes. . In addition, by determining the intermediate position of the vehicle to pass on the two parking frame lines, and finding and connecting the route from the current position of the vehicle to the intermediate position and the route from the intermediate position to the parking position, A route for moving the vehicle to the position may be calculated.

  As shown in FIG. 12, even when only two parking support sheets 40 arranged in the short direction of the parking area are detected, from the same idea as in FIG. A route passing through the sandwiched parking frame line is calculated.

  However, in this case, even if the route is calculated without darkness, the route passing on the parking frame line sandwiched between the two parking support sheets 40 is not calculated. The vehicle position at which the center position of the vehicle is in contact with the intermediate point of the parking frame line to be passed is determined as the intermediate position. Then, a route from the current position of the vehicle to the intermediate position and a route from the intermediate position to the parking position are obtained and connected to calculate a route for moving the vehicle to the parking position. If the vehicle cannot be moved from the current position of the vehicle to the intermediate position by simply reversing the vehicle, a route including turn-back may be calculated. A method for calculating a route including switching is also known from, for example, Japanese Patent Application Laid-Open No. 2010-18074, and the description thereof is omitted.

  Further, as shown in FIG. 14, when only one parking support sheet 40 is detected, it is unclear which of the parking frame lines can be passed. Therefore, as shown in FIG. A plurality of different routes are calculated and displayed as the route followed by the vehicle before reaching the parking position in the area. In the example shown in FIG. 15, two different types of routes are displayed on one screen, but three or more types of routes may be displayed, or each route may be displayed on a separate screen. . Even when two or more parking assistance sheets 40 are detected, a plurality of different routes may be calculated as in the example shown in FIG.

  For example, if there are obstacles (other vehicles, pillars, walls, etc.) at a position adjacent to the parking area of the vehicle, depending on the route to the parking area, the vehicle may come into contact with the obstacle when moving. Can also happen. In this regard, as described above, if a plurality of routes are calculated and displayed, the possibility of presenting a route that can safely park the vehicle can be increased even if there are obstacles. The use opportunity of parking control by a person increases, and more effective parking assistance can be performed.

  In this manner, when the parking area and route calculation unit 24 calculates the parking area and route, the calculation result is output to the image generation unit 23. The image generation unit 23 uses the camera image captured by the CCD camera 10 and the parking region and route calculated by the parking region and route calculation unit 24 to display an actual image in which the parking region and the route are superimposed and / or Alternatively, a bird's-eye view image is generated and output to the display device 30.

  When the parking area defined by the parking frame line and the route to the parking area are displayed on the display device 30, the driver of the host vehicle automatically moves the vehicle to the parking position by an operation switch (not shown). When the execution of the parking control to be moved is instructed, the image processing ECU 20 outputs information related to the parking area and the route to the parking control ECU 50, and parking control is started.

  When a plurality of routes are displayed on the screen of the display device 30, the driver performs a switch operation for instructing execution of parking control while selecting any one of the routes.

  When the execution of the parking control is instructed, the parking control ECU 50 detects the behavior of the vehicle (speed, steering angle, etc.) by various sensors (not shown) so that the vehicle moves along the route instructed by the driver. Control brakes, engines, steering, etc. Therefore, a control signal is output to the brake ECU 60, the engine ECU 70, the steering ECU 90, etc. that control the respective devices. If the instructed route includes turning back, the parking control ECU 50 also outputs a control signal to the transmission ECU 80 to control the shift position. Since such control for automatically moving the vehicle according to the set route is also known, further explanation is omitted.

  The parking assistance system according to the second embodiment described above can be implemented in combination with the parking assistance system according to the first embodiment. That is, when the driver tries to park the vehicle, first, the parking assist system according to the second embodiment displays the parking area and the route, and the driver performs the driving operation himself to park the vehicle, The driver selects whether the vehicle is automatically moved to the parking position by the parking control. And when a driver chooses to perform driving operation himself and parks a vehicle, parking assistance by the parking assistance system by a 1st embodiment is performed, and when parking control is chosen, a 2nd embodiment The parking control may be executed by the parking support system.

10 CCD camera 20 Image processing ECU
30 Display device
40 Parking support sheet 100 Parking support system

Claims (14)

A parking assistance system that assists a driving operation for parking when a driver of a vehicle parks the vehicle in a desired area,
Marking patterns provided on the road surface to determine the positions of at least two intersecting parking frame lines as parking frame lines indicating areas where the vehicle should be parked,
A photographing means mounted on the vehicle for photographing an image around the vehicle;
In a peripheral image photographed by the photographing means, a position specifying means for searching for the landmark pattern and specifying its position;
Based on the position of the mark pattern specified by the position specifying means, the positions of two intersecting parking frame lines are determined, and the positional relationship between the positions of the parking frame lines and the vehicle is notified to the vehicle driver. An informing means ,
The parking pattern is placed at a corner portion of an area where the vehicle is to be parked, and indicates a corner position and two parking frame lines extending from the corner position .   The parking assistance system according to claim 1, wherein the mark pattern is drawn on a movable sheet placed on a road surface. When the seat on which the mark pattern is drawn is placed at each of two adjacent corner portions of the area where the vehicle is to be parked, the notification means relates to a parking frame line between the corner portions on which the seat is placed. The parking support system according to claim 2 , wherein the position of the parking frame line is determined so as to connect the corner positions indicated by the mark patterns of the respective sheets. The notification means, the distance between the vehicle and the parking frame line is below a predetermined threshold value, according to any of claims 1 to 3, characterized in that a warning to the driver of the vehicle Parking assistance system. The informing means has a display device provided in the passenger compartment, displays a vehicle peripheral image based on the image photographed by the photographing means on the display device, and is determined based on the position of the mark pattern. The parking assist system according to any one of claims 1 to 4 , wherein a virtual parking frame line is superimposed on the vehicle periphery image and displayed at a position of the frame line. The parking support system according to claim 5 , wherein the notification unit obtains the shortest distance from the vehicle to the virtual parking frame line and displays the obtained shortest distance on the vehicle peripheral image. The parking according to claim 5 or 6 , wherein the notification means displays a virtual corner pole in a superimposed manner at a position corresponding to an intersection of two intersecting parking frame lines in the vehicle periphery image. Support system. The parking support system according to claim 7 , wherein the notification unit changes a display form of the corner pole according to a degree of approach between the vehicle and the corner pole. The informing means has a display device provided in the passenger compartment, determines a rectangular parking area based on at least the positions of two intersecting parking frame lines, and a path followed by the vehicle until it fits in the parking area And, as the positional relationship between the position of the parking frame line and the vehicle, displays the parking area and the route of the vehicle until it fits in the parking area on the display device,
2. A parking control unit that performs parking control of the vehicle so that the vehicle automatically moves according to a route displayed by the notification unit based on an instruction from a driver of the vehicle. The parking assistance system according to claim 3 . The mark pattern is arranged at a corner portion of the rectangular parking area, and when a plurality of mark patterns located at both ends of each side of the parking area are searched by the position specifying unit, The parking support system according to claim 9 , wherein the notification unit calculates a route that passes on a side between the plurality of searched landmark patterns. The informing means calculates and displays a plurality of different routes as a route followed by the vehicle before it fits in the parking area,
The parking control unit, from among a plurality of paths that are displayed, when the primary path is selected by the driver, according to claim 9, characterized in that the parking control according to the selected route Parking assistance system. The parking route according to any one of claims 9 to 11 , wherein a route that the vehicle follows until it falls within the parking area is superimposed on a vehicle peripheral image based on an image photographed by the photographing means. Support system. The notification means includes
Having a bird's-eye image generation means for generating a bird's-eye image obtained by bird's-eye conversion of the image photographed by the photographing means;
Examples vehicle peripheral image, the image taken by the image pickup means, the bird's-eye image and a bird's-eye image generated by the generating means and displaying side by side on the display device according to claim 5 to claim 8 and claim Item 13. A parking assistance system according to any one of items 12 . The notification means includes
Having a bird's-eye image generation means for generating a bird's-eye image obtained by bird's-eye conversion of the image photographed by the photographing means;
As the vehicle peripheral image, an image photographed by the photographing unit and a bird's eye image generated by the bird's-eye image generation unit are switched in accordance with an instruction from a vehicle driver and displayed on the display device. The parking support system according to any one of claims 5 to 8 and claim 12 .

Images