JP2015096411A - Parking support system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a parking support system which makes a vehicle mounted with an on-vehicle camera and an automatic drive unit perform smooth automatic travel in a parking facility including a slope road surface.SOLUTION: When a vehicle 100 performs automatic travel in a parking facility, an automatic drive unit 57 acquires parking support map information including slope road surface information in the parking facility from a parking support control device 120 of a parking facility side terminal. When determining that a travel position of the own vehicle 100 reaches a slope road surface position (Step S102: YES), the automatic drive unit 57 converts coordinates of a slope road surface Pb in a captured image acquired through an on-vehicle camera 20 into the coordinates of a flat road surface Pa on the basis of the parking support map information including the slope road surface information, converts the coordinate-converted captured image into a planar view image, and makes the vehicle 100 perform the automatic travel on the slope road surface.

Description

  The present invention includes an automatic driving unit mounted on a vehicle and automatically driving the vehicle, and a parking facility terminal (parking support terminal, which can communicate with the automatic driving unit and is provided separately from the vehicle). A parking assistance system comprising a parking assistance device).

  Recently, as shown in Patent Document 1, the management center on the parking facility side automatically provides a travel route to an empty parking position in a parking lot for a vehicle having an automatic driving function that has transmitted vehicle state information. A parking lot management apparatus has been developed that determines and guides the vehicle to an empty parking position using the automatic driving function.

  In the parking lot management apparatus according to Patent Document 1, there has been proposed an apparatus and a method for automatically parking a vehicle at a parking position in the parking lot by performing timely communication between the vehicle and the management center. In particular, the management center uses a plurality of fixedly arranged cameras, so-called surveillance cameras (also referred to as infrastructure cameras), from the images of the entire parking facility, and the parking conditions (whether there are free parking spaces, etc.) Alternatively, it is disclosed that the state of the vehicle (running, stopping, parking, etc.) is recognized.

  However, such a parking facility equipped with a plurality of infrastructure cameras has a high installation cost and a maintenance cost, and there is a difficulty in spreading the parking facility.

  In Patent Document 2, a driver performs an accelerator and a brake operation without using an infrastructure camera, and a steering operation is automatically performed by a steering control electronic control unit using captured images of a front camera, a back camera, a side camera, and the like. A parking assistance device that assists driving operation when a vehicle to be parked at the target parking position is disclosed. In this Patent Document 2, information indicating the slope of the road surface of the parking position is transmitted from the parking lot server as the parking section characteristic information and the control parameter, and the vehicle receives the driving torque when the vehicle moves backward according to the slope. It is disclosed that adjustment can be made as appropriate.

JP 2011-54116 A ([0021], [0024], [0029]) JP 2010-30427 A ([0038], [0051], [0052], [0057], [0066])

  By the way, when using the in-vehicle camera to obtain the vehicle surroundings including the road surface in the parking facility without using the infrastructure camera, and automatically traveling on the road surface in the parking facility, the image acquired (captured) by the in-vehicle camera. Is converted into a planar view image, and automatic running processing is performed based on the image information subjected to the viewpoint conversion.

  However, when the road surface is an inclined road surface having a gradient, when the vehicle reaches a position where the inclined road surface exists from a flat (planar) road surface, the inclined road surface having the gradient in the image information converted from the viewpoint is converted. As a result, the image is distorted, so that it is not possible to continue automatic driving with accurate image information.

  The present invention has been made in consideration of such problems, and a vehicle equipped with an in-vehicle camera and equipped with an automatic driving unit is a road surface from an even road surface to an inclined road surface, an inclined road surface, and further from an inclined road surface to a flat road surface. It is an object of the present invention to provide a parking support system that enables smooth automatic driving on a road surface with accurate image information.

  A parking support system according to the present invention includes a vehicle-mounted camera and an automatic driving unit, and a vehicle having a vehicle position detection function automatically travels in a parking facility while acquiring a vehicle surrounding situation using the vehicle-mounted camera. When the vehicle automatically travels within the parking facility, the automatic driving unit acquires the information on the inclined road surface in the parking facility from the terminal on the parking facility side, and the traveling position of the own vehicle is on the inclined road surface. When it is determined that the vehicle has arrived, the inclined road surface in the captured image acquired by the in-vehicle camera is coordinate-converted into a flat road surface based on the inclined road surface information, and the inclined road surface is automatically traveled.

  According to the present invention, when a vehicle having an in-vehicle camera and an automatic driving unit and having a self-vehicle position detection function automatically travels in the parking facility, the road surface information in the parking facility is obtained from the terminal on the parking facility side. When it is determined that the traveling position of the vehicle has reached the inclined road surface, the inclined road surface is obtained by coordinate-converting the inclined road surface in the captured image acquired by the in-vehicle camera based on the inclined road surface information to a flat road surface. Thus, the road surface and the inclined road surface extending from the flat road surface to the inclined road surface, and the road surface extending from the inclined road surface to the flat road surface can be automatically and smoothly driven by accurate image information.

  In this case, it is preferable that the inclined road surface information includes position information where the inclined road surface exists and gradient information of the inclined road surface.

  The inclined road surface information includes obstacle three-dimensional arrangement information physically connected to the inclined road surface, and when the coordinate conversion is performed on the inclined road surface in the captured image to the flat road surface. The obstacle related to the obstacle three-dimensional arrangement information in the photographed image is coordinate-converted and connected to the flat road surface so as to automatically travel on the inclined road surface, thereby being physically connected to the inclined road surface. It is possible to drive automatically by accurately recognizing obstacles.

  In this case, the obstacle three-dimensional arrangement information includes at least one information of curb information physically connected to the inclined road surface, wall information, and roof information on the inclined road surface. .

  In addition, an inclined road surface automatic running algorithm that automatically transforms an inclined road surface in a captured image acquired by an in-vehicle camera based on known inclined road surface information to a flat road surface and automatically travels on the inclined road surface is arranged on the flat road surface. A flat non-contact charging power receiving device including a secondary coil disposed at the lower part of the vehicle with respect to the position of a stepped object, such as a non-contact charging power feeding unit provided with a primary coil, whose surface is substantially parallel to a horizontal plane. The present invention can also be applied to a case where automatic running parking is performed in order to align the positions of the parts. In this case, the marker on the surface of the stepped object such as the non-contact charging power supply unit is assumed to be on the road surface immediately below the thickness, and the coordinate conversion is performed so that the non-contact charging power supply unit or the like The position of the non-contact charging power receiving unit disposed under the vehicle can be matched with the position of the stepped object so that automatic parking can be performed accurately.

  According to the present invention, when a vehicle equipped with an in-vehicle camera and equipped with an automatic driving unit and having a self-vehicle position detection function automatically travels in the parking facility, the slope surface in the parking facility from the parking facility side terminal. When the information is acquired and it is determined that the traveling position of the vehicle has reached the inclined road surface, the inclined road surface in the captured image acquired by the in-vehicle camera is coordinate-converted to a flat road surface based on the inclined road surface information, Since the vehicle automatically travels on the inclined road surface, the road surface from the flat road surface to the inclined road surface and the road surface from the inclined road surface to the flat road surface can be smoothly and automatically traveled with accurate image information.

It is a schematic plan view which shows the schematic position of the various sensors attached to the vehicle which can be automatically driven used for the parking assistance system which concerns on this embodiment. It is a block diagram which shows the parking assistance system which concerns on this embodiment. It is a schematic plan view showing an example of a parking facility. It is a recognition schematic diagram of a vehicle by an infrastructure camera. It is explanatory drawing of the picked-up image by each infrastructure camera in the state of FIG. It is explanatory drawing of the detection of the movement obstruction by an infrastructure camera. It is a flowchart which shows each process of a parking assistance control apparatus and a vehicle. FIG. 8 is a detailed flowchart of the initial process of step S1 in the flowchart of FIG. In the flowchart of FIG. 8, it is a flowchart corresponding to the parking facility of the modification of the parking availability determination process to the empty parking position space of step S1C. It is explanatory drawing of the movement path | route from the warehousing position to the parking position in the parking facility which concerns on this embodiment. FIG. 8 is a detailed flowchart of the vehicle recognition / tracking process in step S <b> 6 in the flowchart of FIG. 7. It is explanatory drawing which shows the state in which the vehicle was located in the parking position on the travel locus. It is explanatory drawing of the parking assistance by 2 units | sets simultaneous cooperative automatic driving | running | working. It is explanatory drawing of the parking assistance including the obstacle avoidance assistance by two vehicle simultaneous automatic driving | running | working. It is explanatory drawing of the relative positional relationship of the vehicle-mounted camera and virtual camera with respect to the road surface marker of the parking assistance system which concerns on other embodiment. FIG. 16A is an explanatory diagram of a road surface marker image viewed from the in-vehicle camera, and FIG. 16B is an explanatory diagram of a planar view image of the road surface marker viewed from the virtual camera. It is explanatory drawing of the distance error which arises in the estimated position of the said road surface marker when image | photographing the road surface marker drawn on the inclined road surface and carrying out reverse projection conversion with the vehicle-mounted camera on the road surface of a horizontal surface. It is principle explanatory drawing which removes the distance error. It is explanatory drawing of the parking assistance map information regarding the road surface containing an inclined road surface. It is a flowchart with which operation | movement description of the parking assistance system which concerns on other embodiment is provided. It is explanatory drawing of the apparatus and positional relationship of a vehicle-mounted camera and a virtual camera with respect to the road surface marker drawn on the ground pad for non-contact charge which is a level | step difference which concerns on the modification of other embodiment. It is principle explanatory drawing which eliminates the said distance error while explaining the distance error which arises in the estimated position of the road surface marker drawn on the ground pad for non-contact charge with a vehicle-mounted camera.

  Hereinafter, a preferred embodiment (another embodiment to be described later) of the parking assistance system according to the present invention will be given and described in detail with reference to the accompanying drawings.

  In this embodiment, for convenience of understanding, a parking support system using an infrastructure camera, which is a technology related to the parking support system according to the present invention, will be described.

  FIG. 1 is a schematic plan view showing schematic positions of various sensors attached to a vehicle 100 capable of automatic driving used in the parking assist system 14 (see FIG. 2) according to this embodiment. FIG. 2 is a block diagram showing the parking support system 14 according to this embodiment. FIG. 3 is a schematic plan view showing a parking facility 12 as an example.

  In FIG. 1, an in-vehicle camera (front camera) 10 (photographing device) for front photographing is installed between a rearview mirror and an in-car roof during a front windshield of a vehicle 100, a tailgate on the rear surface of the vehicle 100, and the like. An in-vehicle camera (rear camera) 20 (imaging device) for rearward photography is installed in the vicinity of the center of the outer surface of the camera. The vehicle surrounding image including the road surface 104 (FIG. 3) in front of the vehicle can be captured by the in-vehicle camera 10 at a wide angle. Moreover, the vehicle periphery image including the road surface 104 (FIG. 3) behind the vehicle can be taken by the in-vehicle camera 20 at a wide angle. The in-vehicle camera 10 may be installed outside the passenger compartment such as the center of the front grill on the front surface of the vehicle.

  A yaw rate sensor 22 is installed near the center of gravity of the vehicle 100. By integrating the output (yaw rate: angular velocity) of the yaw rate sensor 22, the direction of the vehicle 100, that is, the vehicle direction can be detected. For this reason, the yaw rate sensor 22 functions as a vehicle direction sensor. As the vehicle direction sensor, a geomagnetic sensor or various gyros can be used instead of the yaw rate sensor 22.

  A steering angle sensor 28 is installed on a steering shaft (not shown) of the vehicle 100. A steering angle (of the vehicle 100) of the front wheels 30 (the right front wheel 30R and the left front wheel 30L) is detected by the steering angle sensor 28.

  A wheel speed sensor 26 (right wheel speed sensor 26R and left wheel speed sensor 26L) is installed adjacent to the rear wheel 24 (right rear wheel 24R and left rear wheel 24L) of the vehicle 100. The vehicle speed is detected based on the wheel speed sensor 26, and the moving distance is detected in consideration of the diameters of the front and rear wheels 30, 24. Therefore, the wheel speed sensor 26 also functions as a distance sensor.

  A GPS sensor 32 (position detection sensor) is installed on a dashboard or the like (not shown). The position (latitude, longitude, height) of the vehicle 100 is detected by the GPS sensor 32. The GPS sensor 32 functions as a position sensor of the vehicle 100. As the position sensor, instead of the GPS sensor 32 or in combination with the GPS sensor 32, a traveling distance and a traveling direction at a certain distance or every certain time. Can be detected by the wheel speed sensor 26 and the yaw rate sensor 22 (position detection sensor), and the vehicle position from the coordinate origin can be calculated by so-called inertial navigation. In the case of inertial navigation, a start point position, for example, a warehousing position (a warehousing space or a warehousing space) 102 (either a warehousing position 102A or a warehousing position 102B) shown in FIG. 3 is set as the coordinate origin.

  A shift position sensor 34 (forward / reverse detection sensor) is installed in the vicinity of a shift knob (not shown) of the vehicle 100. Advance or reverse of the vehicle 100 can be detected by the shift position sensor 34.

  As shown in FIG. 2, the various sensors are collectively referred to as a sensor 50. In this case, the position of the vehicle by inertial navigation is calculated by the general ECU 52 connected to each other via an in-vehicle communication line 54 such as an in-vehicle LAN, and inertial navigation is executed. The overall ECU 52, which is a control device that controls the entire vehicle, functions as an in-vehicle parking support control device (an in-vehicle parking support unit). The ECU is an electronic control unit (Electronic Control Unit), a computer including a microcomputer, a CPU (Central Processing Unit), a ROM (including EEPROM), a RAM (Random Access Memory), a RAM (Random Access Memory), In addition, it has input / output devices such as A / D converters and D / A converters, a timer as a timekeeping unit, etc., and various function realizing units by the CPU reading and executing programs recorded in the ROM (Function realization means), for example, functions as a control unit, a calculation unit, a processing unit, and the like.

  The overall ECU 52 is electrically connected to the automatic operation ECU 56 and is connected to the steering control ECU 58, the brake ECU 60, and the driving force ECU 62 through the in-vehicle communication line 54. The general ECU 52 and the automatic operation ECU 56 constitute the automatic operation unit 57, but they may be combined into one ECU.

  The storage unit of the automatic operation ECU 56 is provided with a vehicle type storage unit 64 that stores the type of the vehicle 100, and the vehicle type storage unit 64 includes the vehicle 100 corresponding to the circumscribed cuboid (rectangular model) of the vehicle 100. The dimensions (full length, full width, and full height), wheelbase length, and tread width are stored in advance as minimum information that defines the type of vehicle 100. The minimum turning radius of the vehicle 100 may be included in the minimum information.

  In order for the parking support control device 120 installed in the vicinity of the entrance 116 of the parking facility 12 to acquire the type of the vehicle 100 (described later), the size (full length, full width, full height) of the vehicle 100 (vehicle dimensions). ), The wheelbase length, the tread width, and the like may be stored directly in the vehicle type storage unit 64 of the automatic operation ECU 56. You may memorize | store the manufacturer name of a vehicle, the name of a vehicle, and the grade of a vehicle for obtaining such information indirectly in the vehicle classification memory | storage part 64 as a vehicle classification. Using the vehicle identification information including the vehicle manufacturer name, vehicle name, and vehicle grade, the parking assistance control device 120 can transmit the data from the so-called big data on the Internet or the corresponding server device via the Internet. The size (full length, full width, full height) (vehicle dimensions), wheel base length, tread width, minimum turning radius, and the like of the vehicle 100 can be acquired.

  The types of the vehicle 100 include the size of the vehicle 100 (full length, full width, full height) (vehicle dimensions), wheelbase length, tread width, minimum turning radius, front / rear overhang, maximum weight, and License plate information {plate size, display of registration number on plate (letters, numbers, etc.)} may be included. The license plate information can be recognized and acquired by the parking support control device 120 from a captured image of the infrastructure camera 106 (described later).

  The steering control ECU 58 controls the steering mechanism (not shown) of the vehicle 100 to control the steering angle of the front wheels 30 in this embodiment. The brake ECU 60 controls the behavior of the vehicle 100 including stoppage by independently controlling all four wheels, the front and rear wheels 30 and 24, independently of each other.

  The driving force ECU 62 controls the driving of an unillustrated engine and / or motor and applies the rotational driving force to the front wheels 30 and / or the rear wheels 24 of the vehicle 100 through the transmission or directly, thereby causing the vehicle 100 to travel. I do.

  Information necessary for the sensor 50 is shared by the general ECU 52, the steering control ECU 58, the brake ECU 60, and the driving force ECU 62.

  The overall ECU 52 is electrically connected to the automatic operation ECU 56.

  As shown in FIG. 3, in this embodiment, the parking facility 12, which is an area (area) in which the vehicle 100 is automatically driven, has, for example, a road surface 104 of several tens of meters, and is provided at four corners of a substantially rectangular road surface 104. An infrastructure camera (surveillance camera) 106 (106a, 106b, 106c, 106d), which is a wide-angle fixed camera, is installed.

  As described above, in this embodiment, parking support control processing mainly using the infrastructure camera 106 will be described, and in other embodiments described later, the vehicle-mounted cameras 10 and 20 that do not use the infrastructure camera 106 are used. The parking assistance control process will be described.

  The infrastructure camera 106 is attached so that the line-of-sight (optical axis) of each of the infrastructure cameras 106 a to 106 d faces the lower side on the diagonal line of the substantially rectangular road surface 104, and each infrastructure camera 106 is attached to the entrance 116 of the parking facility 12. (116A, 116B) is configured and installed and adjusted so that a substantially entire area of the road surface 104 including (116A, 116B) can be photographed. In other words, all the moving bodies including the vehicle 100 in the parking facility 12 including the entrance 116 of the parking facility 12 can be photographed (wide-angle photographing) by the infrastructure cameras 106a to 106d, and can be tracked or monitored as described later. It is like that.

  The infrastructure camera 106 is electrically connected to a parking assistance control device 120 such as a boarding / exiting terminal (see FIG. 2), and is taken with respect to a captured image 108 (described later with reference to FIG. 5) captured by the infrastructure camera 106. Various types of image processing and the like are executed by the parking assistance control device 120.

  For example, as shown in FIG. 4, when the vehicle 100 arranged on the road surface 104 of the parking facility 12 is photographed, as schematically shown in FIG. 5, the infrastructure cameras 106 a, 106 b, 106 c, and 106 d are corresponded. Photographed images (also referred to as vehicle extraction images) 108a, 108b, 108c, and 108d are extracted, respectively, and image processing of the parking assistance control apparatus 120 from the photographed images 108 (108a to 108d), here, a known interframe difference method. For example, the captured image (vehicle image) 110 (110a, 110b, 110c, 110d) obtained by detecting the vehicle 100 as a moving body and cutting out the vehicle 100 can be acquired. Not only the inter-frame difference method but also a photographed image of the road surface 104 (referred to as a reference photographed image) including the entrance 116 where no moving body such as the vehicle 100 is present with the infrastructure camera 106 under a prescribed light quantity. In addition, it is possible to cut out (detect) the vehicle 100 or the like that is a moving body from a difference image (an image based on the difference method) from the reference captured image after correcting the amount of light of the captured image 108 when the moving body enters. The light amount correction can be executed by the infrastructure camera 106 or the parking assistance control device 120.

  When the vehicle 100 is cut out, the parking support control device 120 sets a circumscribed cuboid representing the size of the vehicle 100 in order to detect (recognize and track) the type of the vehicle 100, and sets the vehicle dimensions (full length and full width). And the total height) is detected. At the same time, the tire wheel and the tire are detected, and the wheel base length and the tread width are obtained. Get back and forth overhangs as needed. When the vehicle 100 is located at the entrance 116 or the storage position 102, the vehicle dimensions (full length, full width, full height) representing the size of the vehicle 100, the wheel base length, and the tread width are minimized. The field of view of the infrastructure camera 106 is adjusted in advance so that it can be always acquired as information and stored in its own storage unit 118.

  At that time, on the road surface 104 excluding the entrance 116 and the warehousing position 102, even if the circumscribed cuboid cannot be detected, at least one of the upper surface, the side surface, the front surface, and the rear surface of the vehicle 100 (one surface view such as a top view) is present. Since detection is possible, the vehicle 100 that automatically travels from the warehousing position 102 to the parking position (also referred to as a parking section or parking position space) 121 to 126 can be tracked based on the detected one surface and wheelbase length or tread width.

  As shown in FIG. 5, in the captured image 110a by the infrastructure camera 106a, the right front portion and the right side surface of the vehicle 100 can be confirmed (tracked). From the captured image 110a, the shape of the tire wheel, A feature point pattern (also referred to as a feature point or feature pattern) indicating the type of the vehicle 100 such as the wheel base length between the front wheel 30 and the rear wheel 24, the front / rear overhang, and the distance from the front emblem to the tire wheel of the front wheel 30R. .) (Type of vehicle 100) can be extracted.

  In the captured image 110b by the infrastructure camera 106b, the rear and rear upper surfaces of the vehicle 100 can be confirmed (tracked). From the captured image 110b, the shape of the reflector embedded in the rear bumper by the parking assist control device 120, and between the reflectors A feature point pattern of the vehicle 100 (type of the vehicle 100) such as the distance, the distance between the reflector and the emblem, and the tread width between the rear wheels 24R and 24L can be extracted.

  In the captured image 110c by the infrastructure camera 106c, the front portion of the vehicle 100 can be confirmed (tracked), and the tread width between the front wheels 30R and 30L, the front emblem and the front wheel 30 can be checked from the captured image 110c by the parking assist control device 120. The feature point pattern of the vehicle 100 (type of the vehicle 100) such as the distance to the vehicle, the shape of the front lamp, and the distance between the emblem and the license plate can be extracted.

  In the captured image 110d of the infrastructure camera 106d, the left front portion and the left side surface of the vehicle 100 can be confirmed. From the captured image 110d, the parking assist control device 120 determines the tire wheel shape, the wheelbase length, and the front wheel from the front emblem. A feature point pattern (vehicle type) of the vehicle 100 such as a distance to a 30 L tire wheel can be extracted.

  Further, for example, as shown in FIG. 6, a moving obstacle 112 for the vehicle 100 (this example) is clearly different from the external shape of the vehicle 100 stored in advance in the road surface 104 of the parking facility 12 such as a ball or a person. In this example, when a moving obstacle 114 (a person in this example) enters, parking assistance control is performed for these moving obstacles 112 and 114 by the above-described difference method or the like based on the captured image 108 of the infrastructure camera 106. It can be detected (recognized) by the device 120.

  On the road surface 104, a rectangular white line area indicating the entrance (also referred to as an entrance position or entrance position space) 116 of the parking facility 12, an entry position (also referred to as an entry compartment or entry position space) 102 (102A, 102B). In addition to the rectangular white line section indicating the square, the rectangular white line section indicating the parking positions 121 to 126 and the T-shaped white line guideline 128 are drawn, and position information (coordinate information or (Also referred to as map information) is stored in advance in the storage unit 118 of the parking assistance control device 120.

  Furthermore, in the storage unit 118 of the parking assistance control device 120, the type of the vehicle 100 stored at the storage positions 102A and 102B {the size of the vehicle 100 (full length, full width, full height) (vehicle dimensions), and wheelbase length And each movement route (target trajectory) corresponding to the tread width (precisely, the tread width of the steered wheel, but usually the front wheel 30)} is created and stored in advance. The minimum turning radius is set (stored) or calculated (predicted) in advance based on the wheelbase length and the tread width. The storage unit is not the storage unit 118 of the parking support control device 120 but a storage unit of an external server (external distribution unit) connected to the parking support control device 120 via a communication line, and is stored in this storage unit. And may be read out.

  The parking assist control device 120 is obtained by the captured images 108 and 110 of the infrastructure camera 106, the vehicle 100 located at the entrance 116 and the entry position 102, and the cuboid of the vehicle 100 existing at any position in the road surface 104. Detect (track) the vehicle 100 by comparing the size (full length, full width, full height) or one side of the circumscribed cuboid, the wheelbase length, and the tread width with the type of the vehicle (so-called pattern matching). Can do.

  The parking assistance control device 120 of the parking facility 12 and the automatic driving ECU 56 of the vehicle 100 transmit information by bidirectional wireless communication such as DSRC method (narrow region wireless communication method).

  The parking assistance system 14 according to this embodiment basically has the above-described configuration or action. Next, the parking processing system 14 according to the above-described embodiment and the parking process 12 on the entrance 116 and the parking position 102 are parked. The automatic travel parking operation to the positions 121 to 126 will be described in detail based on the flowchart shown in FIG.

  In step S1, an initial process is performed by the parking assistance control device 120.

  FIG. 8 shows a detailed flowchart of the initial process.

  In step S1A, based on the photographed image 108 from the infrastructure camera 106, the parking support control device 120 determines whether or not the vehicle 100 has entered (positioned) the entrance 116 or the warehousing position 102.

  When the parking assistance control device 120 detects that the vehicle 100 has entered the entrance 116 or the storage location 102 from the captured image 108 of the infrastructure camera 106 (step S1A: YES), the type of the vehicle 100 is recognized in step S1B. The

  The recognition of the type of the vehicle 100 can select either one of the two methods described below.

  Vehicle type recognition method 1: When the captured image 108 obtained by photographing the vehicle 100 is received from the infrastructure camera 106, the parking assistance control device 120 transmits a request signal for inquiring the type of the vehicle 100 to the automatic driving unit 57 of the vehicle 100. . When the automatic driving unit 57 receives the request signal, the type of the vehicle 100 stored in the vehicle type storage unit 64 {the size of the vehicle 100 corresponding to the circumscribed cuboid of the vehicle 100 (the total length, the total width, the total height ), The wheel base length, and the tread width} are transmitted to the parking support control device 120, and the parking support control device 120 recognizes the type of the vehicle 100 by receiving and stores it in the storage unit 118.

  Vehicle type recognition method 2: Based on the captured image 108 received from the infrastructure camera 106, a circumscribed cuboid of the vehicle 100 is set from the captured images 108, 110, and the size of the vehicle 100 (full length, full width, and ), The wheel base length, the tread width, and the license plate information are detected from the captured image 110 to recognize the type of the vehicle 100 and store it in the storage unit 118.

  After the type of the vehicle 100 is recognized in step S1B, it is determined in step S1C whether the vehicle 100 can be parked in the empty parking position space based on the recognized type of the vehicle 100.

  If it is determined in step S1C that the vehicle 100 is not a vehicle that can be parked in the empty parking position space (step S1C: NO), the speaker of the parking support control device 120 that parking is not possible in step S1D. The driver of the vehicle 100 is notified from a notification unit such as the above. In addition, when informing, it is also possible to make an announcement from an in-vehicle speaker (sound generating device) (not shown) in the vehicle 100 through the automatic driving unit 57 or to notify the in-vehicle display to that effect.

  The flowchart of FIG. 9 is a flowchart of a modification of the determination process (“whether parking is possible in an empty parking position space”) in step S1C applied to a parking facility different from the parking facility 12 illustrated in FIG.

  A parking facility (not shown) to which the flowchart of this modification is applied has a plurality of large parking positions on the front side when the vehicle 100 is located at the entrance or the entry position, as viewed from the current position of the vehicle 100. There is a parking lot composed of a space (referred to as a parking lot for a large vehicle), and there is a parking lot (referred to as a parking lot for a small vehicle) composed of a plurality of small parking spaces through narrow paths. Assume the case.

  In this case, whether or not the vehicle 100 can be parked in a large parking position space based on the type of the vehicle 100 in step S1Ca (FIG. 9) after the type recognition processing of the vehicle 100 in step S1B (FIG. 8) described above. Is determined. When it is determined that parking is not possible in the large parking position space (step S1Ca: NO), the notification process of step S1D (FIG. 8) described above is executed.

  On the other hand, if it is determined in step S1Ca that parking is possible in a large parking position space based on the type of vehicle 100 (step S1Ca: YES), small parking is performed based on the type of vehicle 100 in step S1Cb. It is determined whether or not parking is possible in the position space. If it is determined that parking is not possible in the small parking position space (step S1Cb: NO), a parking lot in the large parking position space is selected in step S1Ce.

  If it is determined in step S1Cb that parking is possible in a small parking position space based on the type of vehicle 100 (step S1Cb: YES), the depth of the large parking lot is determined based on the type of vehicle 100 in step S1Cc. It is determined whether or not the narrow road existing in can pass. If it is determined that the narrow road is not allowed to pass (step S1Cc: NO), a parking lot having a large parking position space is selected in step S1Ce. On the other hand, if it is determined in step S1Cc that narrow-path traffic is possible (step S1Cc: YES), a parking lot with a small parking position space is selected in step S1Cd.

  In the flowchart of FIG. 9, the processing results of step S1Cd and step S1Ce correspond to the determination of step S1C: YES in the flowchart of FIG. 8, and in the flowchart of FIG. 9, the determination result of step S1Ca: NO is the flowchart of FIG. Note that this corresponds to the determination result of step S1C: NO.

  Next, in step S1E of FIG. 8, it is determined whether or not the vehicle 100 has entered the warehousing position 102 from the captured image 108 of the infrastructure camera 106. Hereinafter, the automatic parking traveling process from the warehousing position 102 to the parking position spaces 121 to 126 will be described with reference to the parking facility 12 shown in FIG.

  Therefore, for example, when it is detected that the vehicle 100 has entered the warehousing position 102A as shown in FIG. 3 (step S1E: YES), the initial process of step S1 shown in FIG. In step S2, the direction of the vehicle 100 (reverse direction or forward direction) is determined by the parking assistance control device 120 based on the captured image 108 from the infrastructure camera 106 or based on information from the shift position sensor 34 from the automatic driving unit 57. Orientation).

  Next, in step S3, a parking support start signal generated by an operation result such as an automatic parking start button (not shown) installed in the parking support control device 120 being pushed by a driver getting off the vehicle 100 or the like. It is determined by the parking assistance control device 120 whether or not it has been received.

  When the parking support start signal is received (step S3: YES), in step S4, the parking support control device 120 moves to the vacant parking position (here, the parking position 122) selected in step S1C. A route (target trajectory) is created or selected based on the type of the vehicle 100 that is in the storage position 102A.

  Here, for example, as shown in FIG. 10, a movement route (target locus) 130 is created. The travel route 130 is basically calculated as an integrated value of a combination of the steering angle of the vehicle 100 (0 [deg] in a straight line) and the travel distance of the vehicle 100 that maintains this steering angle.

  Next, in step S5, the automatic parking start control device 120 transmits an automatic travel start instruction and information on the travel route 130 (the integrated value) to the automatic operation ECU 56 of the vehicle 100.

  The automatic operation ECU 56 on the vehicle 100 side is in a state of waiting for an automatic travel start instruction in step S31, and when receiving an automatic travel start instruction (step S31: YES), the automatic travel control and the vehicle are performed in step S32. Position / direction detection processing is performed sequentially (every predetermined travel distance or every predetermined time).

  That is, in step S32, the vehicle position is detected or calculated from the result of traveling by inertial navigation based on the outputs of the steering angle sensor 28, the wheel speed sensor 26, and the yaw rate sensor 22, or the sequential reception position of the GPS sensor 32. The calculation of the travel locus by the inertial navigation is executed by the overall ECU 52, and the calculation result is sequentially transmitted to the parking assistance control device 120 via the automatic driving ECU 56. The vehicle direction is detected by the overall ECU 52 based on the shift position (forward position, reverse position) by the shift position sensor 34.

  Next, in step S33, the direction of the vehicle 100 based on the detection result (forward driving direction corresponding to the forward position or backward driving direction corresponding to the reverse position) and the vehicle position are sequentially parked via the automatic driving ECU 56. It is transmitted to the control device 120.

  After the transmission, in step S34, the automatic operation ECU 56 determines whether or not an automatic travel end instruction has been received from the parking assistance control device 120. If not received (step S34: NO), the process proceeds to step S32. Returning, the process of step S32 is continued.

  On the other hand, the parking assistance control device 120 that has detected the automatic travel of the vehicle 100 (travel for automatic parking) by receiving the vehicle position / direction information performs the vehicle recognition / tracking process in step S6.

  FIG. 11 is a detailed flowchart of the vehicle recognition / tracking process in step S6.

  In step S6a, when the parking assistance control device 120 receives (acquires) information on the vehicle position / direction from the automatic driving ECU 56, in the next step S6b, a captured image (in FIG. Based on the captured images 110a, 110b, 110c, 110d, etc. shown), the relative posture of the vehicle 100 (how the vehicle 100 is viewed from each infrastructure camera 106) is determined.

  Next, in step S6c, it is determined whether there is a captured image 110 in which the side surface of the vehicle 100 can be seen from the posture determined in step S6b. If there is a captured image 110 in which the side surface is visible (step S6c: YES), In step S6d, a feature point pattern (tire wheel shape, wheelbase length, etc.) for discriminating the type of the vehicle 100 related to the above-described side surface is extracted from the captured image 110 in which the side surface, which is basically a saddle, can be seen. In step S6e, it is determined by pattern matching whether or not the extracted feature point pattern (type of vehicle 100) matches the previously extracted feature point pattern (type of vehicle 100). If the pattern does not match (step S6e: NO), automatic operation is interrupted in step S6f. When the automatic driving is interrupted in step S6f, the parking assistance control device 120 notifies the maintenance staff and the like to that effect. In this case, the cause of failure is investigated and recovered by so-called offline processing by maintenance personnel. In practice, the determination in step S6c should be detected by any one of the captured images 108, 110 of the four infrastructure cameras 106 regardless of where the vehicle 100 is located on the road surface 104 of the parking facility 12. Therefore, the determination in step S6c is usually positive (established).

  On the other hand, when the feature point pattern (type of vehicle 100) extracted this time matches the feature point pattern (type of vehicle 100) extracted last time in step S6e, the parking assistance control device 120 in step S6g. The current position (current position coordinates) on the road surface 104 of the vehicle 100 is detected on the basis of the current captured images 108 and 110 subjected to pattern matching with reference to the map of the own parking facility 12.

  Next, returning to the flowchart of FIG. 7, in step S7, it is determined whether or not the current position is the parking position 122. When the current position is not the parking position 122 (step S7: NO), in step S8, The current position detected in step S6g is transmitted to the automatic operation ECU 56. At this time, the positional deviation amount (deviation) of the current position with respect to the instructed movement route 130 may be transmitted instead of the current position or together with the current position.

  On the other hand, when the side surface is not visible in step S6c of FIG. 11 (step S6c: NO), it is determined in step S6h whether or not there are captured images 108 and 110 in which the front surface of the vehicle 100 can be seen. When there are the photographed images 108 and 110 that can be seen (step S6h: YES), in step S6i, the feature point pattern (emblem and front wheel 30 from the emblem and the front wheel 30 to the front wheel 30) is captured from the photographed images 108 and 110 in the same manner as in step S6d. Distance, etc.) (type of vehicle 100) is extracted, and whether or not the extracted feature point pattern (type of vehicle 100) matches the previously extracted feature point pattern (type of vehicle 100) in step S6e. Judgment is made by matching, and after that, step S6e: YES, step S6g, and step S7: NO 8 performs position transmission process (Figure 7).

  In step S6h, when the front surface is not visible (step S6h: NO), the rear surface should be visible. Therefore, in step S6j, similar to the processing in step S6d, the captured image 110 of the infrastructure camera 106 is related to the rear surface. A feature point pattern (distance between reflectors, reflector shape, etc.) (type of vehicle 100) is extracted, and in step S6e, the extracted feature point pattern (type of vehicle 100) is extracted last time. It is determined by pattern matching whether or not the type matches, and thereafter, the position transmission process in step S8 is performed through each process of step S6e: YES, step S6g, and step S7: NO.

  Next, the overall ECU 52 that has received the current vehicle position or the deviation amount (deviation) of the vehicle position from the parking assistance control device 120 via the automatic operation ECU 56 again, in step S32, the above-described automatic travel control and vehicle position / Direction detection processing is performed sequentially (every predetermined travel distance or every predetermined time).

  However, in the second and subsequent processes in step S32, a distance correction process and a steering angle correction process for correcting a positional deviation amount (deviation) between the movement route 130 and the current position are performed.

  Thereafter, a vehicle position / direction transmission process is performed in step S33, and a vehicle recognition / tracking process is performed in step S6.

  In this way, when the determination at step S7, that is, when the current position is (positioned) at the parking position 122 on the travel locus, the parking assist control device 120 is automatically set at step S9 as shown in FIG. Send a travel end instruction.

  The automatic operation ECU 56 that has received the automatic travel end instruction in step S34 terminates the automatic travel through the overall ECU 52 in step S35, transmits the fact that the automatic travel has been terminated to the parking support control device 120, and the main switch such as an ignition switch. Turn the switch off.

  In addition, in the parking assistance system 14 according to this embodiment, since all the moving bodies on the road surface 104 of the parking facility 12 can be detected, as shown in FIG. The two vehicles 100, 100 ′ can be parked by automatically traveling (simultaneous two-unit automatic traveling) at the parking positions 122, 125 along the movement paths 130, 132.

  In addition, as shown in FIG. 14, when the vehicles 100 and 100 ′ are discharged from the parking positions 122 and 125 to the storage positions 102 </ b> A and 102 </ b> B, the parking assistance control device 120 is also used even when the movement paths 134 and 136 cross each other. Since the current position of the vehicles 100 and 100 'and the movement paths 134 and 136 are grasped, collision determination (when there is a possibility that the vehicles 100 and 100' partially overlap at the same position at the same time, they may overlap) One vehicle (vehicle 100 or vehicle 100 ′) is stopped until only the other vehicle (vehicle 100 ′ or vehicle 100 ′) is moved until it becomes non-existent, and is stopped in a state where there is no possibility of overlapping. The other vehicle is allowed to travel again, so that interference can be avoided.

  In this case, when the position, moving direction 141, and moving speed of the moving obstacle 142 such as a ball are detected, an interference avoidance process between the movement predicted path of the moving obstacle 142 and the moving paths 134, 136 is performed. It is also possible (obstacle avoidance support control by two-unit simultaneous automatic driving).

  As described above, the parking support system 14 according to the above-described embodiment is disposed in the parking facility 12 and is disposed on the parking facility 12 side with the infrastructure camera 106 as at least one fixed camera that captures the vehicle 100, A parking support control device 120 that receives a captured image 108 transmitted from the infrastructure camera 106 and creates a travel route 130 as vehicle guidance information, and is mounted on the vehicle 100 and receives information on the travel route 130 from the parking support control device 120. And an automatic operation unit 57 that automatically travels the vehicle 100 from the warehousing position 102 to the parking position space 122 based on the received information on the moving route 130.

  The parking support control device 120 recognizes the type of the vehicle 100 when the vehicle 100 is arranged at the entrance 116A or the warehousing position 102A of the parking facility 12, and based on the recognized type of the vehicle 100, the vehicle 100 It is determined whether or not the vehicle 100 can be parked in the parking position space 122 in the vehicle 12.

  According to this embodiment, the parking assistance control device 120 recognizes the type of the vehicle 100 when the vehicle 100 is arranged at the entrance 116A or the storage position 102A of the parking facility 12, and based on the recognized type of the vehicle 100. Since it is configured to determine whether or not the vehicle 100 is a vehicle 100 that can be parked in the parking position space 122 in the parking facility 12, the vehicle 100 is placed in the parking position space 122 in the parking facility 12. Whether or not parking is possible can be determined instantaneously with less processing load.

  In this case, the vehicle 100 further includes a vehicle type storage unit 64 that stores the type of the vehicle 100, and the parking support control device 120 is arranged when the vehicle 100 is arranged at the entrance 116A or the storage position 102A of the parking facility 12. In addition, when recognizing the type of the vehicle 100, when the captured image 108 obtained by photographing the vehicle 100 is received from the infrastructure camera 106 at the entrance 116A or the storage location 102A of the parking facility 12, a request signal for inquiring the type of the vehicle 100 is sent to the vehicle. When the automatic driving unit 57 receives the request signal, the automatic driving unit 57 transmits the type of the vehicle 100 stored in the vehicle type storage unit 64 to the parking assistance control device 120, and parking assistance. The control device 120 determines that the vehicle 100 is a parking facility based on the received type of the vehicle 100. Whether it is a vehicle 100 that can be parked in the parking position the space 122 within 2 or may be determined.

  Alternatively, the infrastructure camera 106 photographs the vehicle 100 when the vehicle 100 is placed at the entrance 116A or the entry position 102A of the parking facility 12, and parks the captured image 108 at the entrance 116A or the entry position 102A of the parking facility 12. The parking support control device 120 that has transmitted to the support control device 120 and has received the captured image 108 at the entrance 116A or the warehousing position 102A of the parking facility 12 recognizes the type of the vehicle 100 based on the received captured image 108, You may make it determine whether it is the vehicle 100 which can be parked in the parking position space 122 in the parking facility 12. FIG.

  In this case, when it is determined that the vehicle 100 is not the vehicle 100 that can be parked in the parking position space 122 in the parking facility 12, the fact is notified from the notification unit of the parking assistance control device 120 and / or the notification unit of the automatic driving unit 57. It is preferable to report from (a vehicle-mounted speaker or vehicle-mounted display of the vehicle 100).

  Furthermore, when the parking assist control device 120 recognizes the type of the vehicle 100 at the entrance 116A of the parking facility 12 or the storage location 102A of the vehicle 100, from the storage location 102A according to the recognized type of the vehicle 100 to the parking location space 122. The movement route 130 may be generated as vehicle guidance information and transmitted to the automatic driving unit 57.

  The types of the vehicle 100 preferably include the size (the total length, the total width, and the total height), the wheel base length, and the tread width of the vehicle 100, but the size of the vehicle 100, the wheel base length, It may include at least one of tread width and license plate information.

  In addition, the parking assist control device 120 continues to recognize the type of the vehicle 100 from the captured image 108 (110) of the infrastructure camera 106 while the vehicle 100 is automatically traveling from the storage position 102A to the parking position space 122. It is preferable to track or monitor whether or not the vehicle 100 is moving along the movement path 130 based on the type of the vehicle 100 that continues.

  Here, the parking assistance control device 120 is based on the vehicle information including the vehicle position and the moving direction and the captured image 108 (110) while the vehicle 100 is automatically traveling from the storage position 102A to the parking position 122. By tracking (feedback control) through the automatic driving unit 57 so that the vehicle 100 moves along the movement route 130, the vehicle 100 can accurately travel and park along the movement route 130.

  Furthermore, the parking assist control device 120 is configured to detect the moving obstacle 142 and (the vehicle 100 ′ for the vehicle 100 or the vehicle 100) by using the captured image 108 (110) while the vehicle 100 is automatically traveling along the movement paths 134 and 136. When the vehicle 100) detects the possibility that the vehicle 100) will enter the movement path 134, 136, it calculates the interference avoidance method between the vehicle 100, 100 'and the moving obstacle 142 (100', 100) and automatically By transmitting to the driving unit 57, avoidance processing such as avoiding interference with the moving obstacle 142 (100 ', 100) by the vehicles 100, 100' mounted with the automatic driving unit 57, for example, stopping and spending time. It can be done smoothly.

  The vehicle 100 further includes an in-vehicle camera 20, and the automatic driving unit 57 stops the vehicle 100 based on the captured images 108 and 110 when the vehicle 100 is stopped at the parking position 122. By configuring, the vehicle 100 can be stopped at the parking position 122 very accurately.

[Other Embodiments]
Next, when the vehicle 100 automatically travels on the road surface 104 in the parking facility 12 instead of the automatic travel of the vehicle 100 using the infrastructure camera 106, the vehicle surface 10 in the parking facility 12 is moved by the in-vehicle cameras 10 and 20. The vehicle surrounding situation is acquired (captured), and the captured image acquired (captured) is converted into a planar view image (planar view image information) by the automatic driving unit 57, and the vehicle 100 automatically travels based on the converted planar view image information. A parking support system based on the technology for processing will be described.

  In this case, the automatic operation unit 57 (in this case, the overall ECU 52) captures images taken by the in-vehicle cameras 10 and 20 when the vehicle 100 is moving forward (in the case of the in-vehicle camera 10 being a front camera) or backward when moving backward. Viewpoint conversion (reverse) while correcting image distortion and the like so as to obtain an image (planar view image) looked down from a virtual viewpoint position of a predetermined height above the vertical direction (in the case of the in-vehicle camera 20 which is a rear camera). Projective transformation).

  Here, with reference to FIGS. 15, 16A, and 16B, an example of viewpoint conversion (coordinate conversion, reverse projection conversion) of a captured image of the in-vehicle camera 20 will be described by taking the in-vehicle camera 20 of the rear camera as an example.

  As shown in FIG. 15, when the road surface 104 is a flat surface (horizontal plane) and the road surface marker 202 made of a square frame drawn on the road surface 104 with a paint or the like and the vehicle 100 are on the same plane, FIG. As shown, in the photographed image 200A (road surface image viewed from the attachment position Ca) captured from the attachment position Ca (center of the photographing lens) of the in-vehicle camera 20, the road surface marker 202 that is actually a square frame has a trapezoidal shape. It looks like a road surface marker image 202A composed of a frame.

  In FIG. 15, a region with a halftone dot indicates a region included in a field angle (shooting field angle) range in which a range captured by the image sensor of the in-vehicle camera 20 is represented by an angle.

  The road surface marker image 202A is a photographed image in the camera video coordinate system Qa (xi, yi) {the origin coordinate Qa (0, 0) is the intersection of the optical axis La of the vehicle-mounted camera 20 and the road surface 104}.

  From the in-vehicle camera 20 in the front-rear direction of the vehicle 100, it is rearward by a predetermined distance Y0 (see FIG. 15), and is improved upward and downward from the vehicle 100. The planar view image is a planar view image 200B shown in FIG. 16B.

  In this case, the planar image 200B after the viewpoint conversion shown in FIG. 16B is the trapezoidal road surface marker image 202A (FIG. 16A) in the road image viewed from the mounting position Ca of the in-vehicle camera 20, and the viewpoint position of the virtual camera 20v. It is converted into a road surface marker image 202B composed of a square frame having a correct shape obtained by back-projection conversion into a road surface image in the virtual camera image coordinate system Qb (xr, yr) viewed from Cb.

  The road surface marker image 202B is a captured image in the virtual camera video coordinate system Qb (xr, yr) {the origin coordinate Qb (0, 0) is the intersection of the optical axis Lb of the virtual camera 20v and the road surface 104}.

  Then, the automatic driving unit 57 is based on a photographed image 200B (referred to as the above-described planar view image or processed image) whose viewpoint has been changed, in addition to a road surface on which the vehicle 100 can travel, a side wall that cannot travel, a curb, and other parking. Recognizing a vehicle and an obstacle such as a hedge (used as a landmark for driving), the vehicle 100 is automatically driven on the road surface 104 capable of traveling while avoiding the obstacle. In this way, the automatic driving unit 57 causes the vehicle 100 to automatically travel from the entrance 116 or the storage position 102 of the parking facility 12 to the target parking position 122, for example.

  However, when the road surface on which the vehicle 100 is to automatically travel includes an inclined road surface instead of a flat road surface, when the viewpoint is changed, the road surface that can actually travel in the vehicle width direction is uniform. However, the traveling road surface in the processed image (plan view image) gradually narrows forward (or rearward) or vice versa, and the automatic driving unit 57 views the vehicle 100 in plan view. It becomes difficult to accurately drive automatically based on the image.

  Next, referring to FIG. 17, a plane (also referred to as a road surface or a flat road surface) Pa on which the vehicle 100 is placed, and a road surface marker M imaged by the in-vehicle camera 20 (represented by dots in FIG. 17). Is a plane (also referred to as a road surface, an inclined surface or an inclined road surface) Pb drawn from the plane Pa parallel to the horizontal plane and the horizontal plane (plane Pa). A problem that occurs in the estimated position of the road surface marker M, that is, a distance error E, occurs when a reverse projection transformation is performed on the assumption that a plane (also referred to as an inclined road surface) Pb that is an inclined plane is on the same plane. The problem that occurs (in the example of FIG. 17, the image taken by the in-vehicle camera 20 appears close to the distance error E) will be specifically described.

  In the example of FIG. 17, for convenience of understanding, the difference between the plane Pa and the inclined plane Pb will be described by limiting to the tilt angle (decline angle, tilt angle) Δθ. Even when there is a change in the angle and roll angle, the same explanation can be made.

  As shown in FIG. 17, the imaging surface 20i of the in-vehicle camera 20 is on a plane perpendicular to the optical axis La of the lens including the focal point f.

  Assuming that the y-coordinate yra of the intersection point of the perpendicular line dropped from the focal point f to the road surface Pa and the road surface Pa is yra = 0, and the tilt angle Δθ of the inclined surface Pb with respect to the plane Pb is a minute angle, FIG. It can be considered that the length of the line segment Yg-yrb (yi1, Pb) is equal to the length of the line segment Yg-yra (yi1, Pb). Therefore, it is desired to correctly recognize the y coordinate yra from yra = 0 of the road surface marker M drawn on the inclined surface Pb as the y coordinate yra (yi1, Pb) {= yrb (yi1, Pb)}.

  In this case, the road surface marker M drawn on the inclined surface Pb is the intersection position of the optical axis La on the imaging surface 20i of the vehicle-mounted camera 20 (distance origin yi = 0 on the upper coordinate yi along the imaging surface 20i). ) To the position of coordinates (distance) yi = yi1 on the imaging surface 20i.

  The angle β is an angle formed by a line Lm (including an extension line) connecting the focal point f and the position of the coordinate (distance) yi1 on the imaging surface 20i and the optical axis La.

In consideration of the focal length F, the formed angle β can be obtained by the following equation (1).
β = arctan (yi1 / F) (1)

Then, if the estimated distance from the origin y coordinate yra = 0 to the road surface marker M drawn on the inclined surface Pb is the height H of the focal point f from the road surface Pa, the following height ( Due to the reverse projection transformation of equation (2), the position coordinate yra (yi1, Pb) regarded as correct is misrecognized as a distance from the position coordinate yra (yi1, Pa) before the length (distance error) E. .
yra (yi1, Pa) = H × tan (θ + β) (2)

That is, a distance error E expressed by the following equation (3) occurs. Note that the angle θ is a known angle (mounting angle of the in-vehicle camera 20) because it is an angle (also referred to as an angle θ) formed with respect to the perpendicular of the optical axis La of the in-vehicle camera 20.
E = yra (yi1, Pb) -yra (yi1, Pa) (3)

  Then, referring to FIG. 18, next, a road surface (plane) Pa where the vehicle 100 exists and a road surface (inclined surface, inclined road surface) Pb on which a road surface marker M imaged by the in-vehicle camera 20 is drawn, In other words, in other words, the vehicle 100 is placed (exists) at a position of a known distance Yg (the known distance will be described later) from the current position (yra = 0) of the vehicle 100. ) When a plane (inclined surface) Pb having a tilt angle Δθ is connected (exists) to a plane (road surface) Pa, on the y coordinate yra along the road surface (plane) Pa on which the vehicle 100 is placed. The position coordinates on the road surface (plane) Pa that are regarded as having almost no error can be obtained from the correct coordinate position yrb (yi1, Pb) on the y coordinate yrb of the inclined surface (plane) Pb without the occurrence of the distance error E. Processing to calculate that it is in yra (yi1, Pb) The reason will be explained.

As shown in FIG. 18, the length L1 of the perpendicular drawn from the focal point f to the plane Pa to the position of the intersection ip1 with the extended line of the inclined surface Pb is L1 = H−ΔH. Here, ΔH is calculated by the following equation (4).
ΔH = Yg × tan Δθ (4)

  The distance Yg from yra = 0 to the position where the inclined surface Pb begins (the position is also Yg) is a known distance based on the parking assistance map information stored in the storage unit 118 of the parking assistance control device 120. Can be sought.

  Next, since the tilt angle Δθ is very small, the length L2 from the focal point f2 to the position of the intersection point ip2 between the perpendicular line extending from the focal point f with respect to the extended line of the inclined surface Pb and the extended line, the focal point f Is assumed to be equal to the length L1 from the focal point f1 to the position of the intersection point ip1 between the perpendicular drawn from the plane Pa to the extended line of the inclined surface Pb (L2≈L1).

Then, on the y coordinate (axis) yrb along the inclined plane Pb, the line from the position of the intersection ip2 to the coordinate position yrb (yi1, Pb) where the road surface marker M is drawn with the position of the intersection ip2 as a zero value (origin) The length yrb (yi1, Pb) of the minute ip2-yrb (yi1, Pb) can be calculated by the following equation (5).
yrb (yi1, Pb) = L2 × tan (θ + Δθ + β)
≒ L1 × tan (θ + Δθ + β)
= (H−ΔH) × tan (θ + Δθ + β) (5)

  The formed angle β can be read from the equation (1), the formed angle θ can be read from the parking assistance map information (details will be described later), etc. Is an angle.

  Since the tilt angle Δθ is very small, the line segments from the intersection ip1 on the inclined plane Pb and the y-coordinate (axis) yrb to the coordinate position yrb (yi1, Pb) are drawn in FIG. And the length of the line segment from the intersection point ip0 on the y coordinate (axis) yra of the road surface Pa to be obtained to the coordinate position yra (yi1, Pb) are assumed to be equal.

That is, the coordinate position yra (yi1, Pb) obtained by projecting the road surface marker M drawn on the inclined surface Pb onto the road surface Pa on which the vehicle 100 is mounted (exists) has a sufficiently small tilt angle Δθ. When it can be seen, the focal point f is projected from the coordinate position yrb (yi1, Pb) representing the length onto the extended line of the inclined surface Pb on which the road surface marker M is drawn, as shown in the following equation (6). It can be seen that it is substantially equal to the value obtained by subtracting the length Δy between the intersection point ip2 and the intersection point ip1. The length Δy can be calculated by the following equation (7).
yra (yi1, Pb) = yrb (yil, Pb) −Δy (6)
Δy = (H−ΔH) × tan (Δθ) (7)

  In practice, in a parking facility (not shown), when the vehicle 100 is guided from the first floor entrance or entry position to the second floor parking position or underground parking position, or from the second floor parking position or underground parking position. When guiding 100 to the first floor exit on the ground (which may be the same as the entrance or the warehousing position described above), it is necessary to automatically run the inclined surface.

  Therefore, in another embodiment of the parking support system according to the present invention, the above-mentioned entrance and storage on the ground of the parking facility 12 are stored in the storage unit 118 (see FIG. 2) of the parking support control device 120 (terminal on the parking facility 12 side). Parking assistance map information (parking assistance map database) necessary for traveling on the route from the position to the parking position on the basement or the second floor is stored in advance.

  The parking assistance map information includes road surface type information indicating whether the traveling road surface in the parking facility 12 is a flat road surface or an inclined road surface, and three-dimensional position information of the start position and the end position of the flat road surface. (Including road surface width information), three-dimensional position information (including road surface width information) of the start position and end position of the inclined road surface connected to the flat road surface, and gradient information of the inclined road surface, Three-dimensional position information of obstacles (side walls, curbs, hedges, ceiling surfaces) connected to the traveling road surface (flat road surface and inclined road surface) is stored in advance. If the start position three-dimensional information and the end position three-dimensional information of the inclined road surface are known, the gradient of the inclined road surface can be calculated by the automatic operation unit 57.

  In practice, it is impractical to accurately express the parameters of a continuously changing plane as map data because the amount of data is enormous. Therefore, in this [other embodiment], as shown in FIG. 19, the road surface 104 in the parking facility 12 (which is different from the map shown in FIG. , Y) = G (1,1), G (1,2),... G (2,1)... (Here, square lattice), and each grid G (x, y) has a single Considering a plane having characteristics (parameters), for each grid G (x, y) (referred to as grid plane or grid plane), characteristic expression (parameter expression) is stored as parking support map information (parking support map database). Stored in the unit 118.

  In this [other embodiment], the height h from the reference position, for example, the warehousing position 102A (see FIG. 3), the inclination (gradient) Δθ in the parallel direction, and the inclination (gradient) Δφ in the meridian direction are used as parameters representing the plane. Is used. That is, each grid G (x, y) is stored in the storage unit 118 as grid characteristics (grid parameters, grid characteristics) G (x, y, h, Δθ, Δφ). That is, the grid characteristic G at the warehousing position 102A can be expressed as G (x, y, h, Δθ, Δφ) = G (0, 0, 0, 0, 0) = G0.

  In addition, on the road surface 104, as shown in FIG. 19, the white line 203 for parking guidance assistance, the division line 204 which divides the parking positions 121a-126a are drawn, and the parking curb 206 may be arrange | positioned. .

  In addition, a non-contact charging ground pad 210 which is a stepped object having a road surface marker 212 drawn on the surface, which will be described later, is disposed at the parking position 126a.

  Next, the operation of [another embodiment] in which an inclined surface is included in the road surface 104 in which the infrastructure is basically configured as described above will be described with reference to the flowchart of FIG. In this [other embodiment], it should be noted that the infrastructure camera 106 illustrated in FIG. 3 may not be installed in the parking facility 12, for example.

  In the initial process of step S101, the vehicle 100 is stopped at a predetermined entrance 116 or storage location 102 (both not shown in FIG. 19) of the parking facility 12, and the parking support control device 120 (a boarding / alighting terminal, parking application) When an automatic parking start button or the like provided in the control device) is pressed, the target parking positions 121 to 126 (FIG. 3) or the target by automatic driving by cooperation between the parking assist control device 120 and the automatic driving unit 57 of the vehicle 100 Parking assistance to one of 121a to 126a (FIG. 19) is started, and the parking assistance control device 120 moves from the entrance 116 or the warehousing position 102 to the target parking position (121 to 126 or one of 121a to 126a). A route (target locus) is set. The start of parking support is not performed by pressing the automatic parking start button, but by operation of a mobile terminal such as a smartphone possessed by the driver of the vehicle 100, or operation of a so-called keyless key or smart entry key of the vehicle 100. Can be substituted.

  At this time, in step S101, the parking support control device 120 further includes a travel support (with target route) parking support map in which the travel route (target route) is set in the parking support map information read from the storage unit 118. The information is transferred to a work storage unit (not shown) of the automatic driving unit 57 of the vehicle 100.

  Next, in step S102, the automatic driving unit 57 refers to the parking support map with moving route transferred and stored in the storage unit for its work, and the vehicle position is, for example, on the ground (1F ) To an underground parking position (inclined road surface start position) or an inclined road surface position (inclined road surface start position) from the ground (1F) to a 2F parking position or the like is determined.

  When it is determined that the slope road surface position (inclined road surface start position) has been reached based on the slope (gradient) (Δθ, Δφ) information of the grid characteristics G (x, y, h, Δθ, Δφ) (step S102: YES) In step S103, the already read slope road surface information (coordinate conversion information) for coordinate conversion is read from the storage unit for its own work.

  When the vehicle position has not reached the inclined road surface position (inclined road surface start position) in the determination of step S102 (step S102: NO) and when the inclined road surface information (coordinate conversion information) is read in step S103, automatic In step S104, the driving unit 57 converts the captured image including the traveling road surface from the in-vehicle cameras 10 and 20 into a planar view image (in the case of the plane Pa, refer to FIGS. 16A and 16B, refer to FIGS. 16A and 16B, the inclined surface Pb from the plane Pa). When the image is taken, refer to FIG. 17 and FIG. 18), and run by inertial navigation based on the plane-view image converted from the viewpoint and the parking assistance map information.

  In this case, as described above, the traveling by inertial navigation is performed by detecting the traveling distance and traveling direction for each fixed control distance or for each fixed control time by the wheel speed sensor 26 and the yaw rate sensor 22, and the coordinate origin (inlet 116 or The vehicle position from the warehousing position 102) is calculated (integrated). Then, the vehicle position is corrected by correcting the difference between the accumulated travel locus (control amount) of the own vehicle position and the movement route (target amount in the target locus). In this traveling by inertial navigation, errors from the coordinate origin are accumulated.

  In the above-mentioned [Embodiment] and [Other Embodiments], a so-called map matching process is performed at a longer distance or a longer time in parallel with the traveling by the inertial navigation.

  That is, in this map matching process, the planar view calculated from the parking assistance map information every fixed control distance longer than the fixed control distance of traveling by the inertial navigation or every fixed control time longer than the fixed control time. Matching (map matching, shape matching) and the like of the specific landmark in the image with the corresponding specific landmark in the planar image of the image taken by the in-vehicle cameras 10 and 20 Is corrected (an image in which the azimuth direction of the parking assistance map is rotated in accordance with the direction of the vehicle).

  It should be noted that other vehicles (moving or parked) that do not exist in the parking support map information and the moving obstacles 112 and 114 (see FIG. 6) are detected based on the images taken by the in-vehicle cameras 10 and 20, and other vehicles are also obstructed. Drive with things.

  Next, in step S105, it is determined whether or not the vehicle 100 has reached the target parking position, and the processing from step S102 to step S104 is repeated until the vehicle 100 reaches the target parking position (step S105: YES).

  As described above, the parking assist system 14 (the infrastructure camera 106 is not necessary) according to the other embodiments described above includes the in-vehicle camera 10 (or the in-vehicle camera 10 and the in-vehicle camera 20), the automatic driving unit 57, and the self-drive unit 57. A vehicle (also referred to as own vehicle) 100 having a vehicle position detection function (may be based on inertial navigation, or may be used in combination with GPS sensor 32 if the radio wave condition is good) is When the vehicle 100 automatically travels in the parking facility 12 in the parking support system 14 (infrastructure camera 106 is not required) that automatically travels in the parking facility 12 while acquiring the surrounding situation, for example, at the entrance of the parking facility 12 In 116 or the warehousing position 102, the automatic driving unit 57 is a parking assistance control device 12 which is a terminal on the parking facility 12 side. The parking assistance map information including the slope road surface information {grid characteristics G (x, y, h, Δθ, Δφ)} in the parking facility 12 is acquired, and the traveling position of the vehicle 100 reaches the slope road surface Pb position. 17 is determined (step S102: YES), for example, when it is determined that the position before the position Yg on the flat road surface Pa is reached in FIG. 17, the slope road surface information {grid characteristics G (x, y, h, [Delta] [theta], [Delta] [phi])} based on the parking assistance map information, the viewpoint is converted to a planar image obtained by coordinate-transforming the inclined road surface Pb and the like in the captured image acquired by the in-vehicle cameras 10 and 20 into the flat road surface Pa, and The vehicle automatically runs on the inclined road surface Pb.

  For this reason, the road surface from the flat road surface Pa to the inclined road surface Pb, the inclined road surface Pb, and the road surface from the inclined road surface Pb to the flat road surface Pa can be smoothly (accurately) automatically traveled with accurate image information.

  In this case, the inclined road surface information {grid characteristics G (x, y, h, Δθ, Δφ)} is obtained from the positional information (x, y, h) where the inclined road surface Pb exists in the parking facility 12 and the inclination. What is necessary is just to consist of the gradient information (Δθ, Δφ) of the road surface Pb.

  The inclined road surface information {grid characteristics G (x, y, h, Δθ, Δφ)} includes obstacle three-dimensional arrangement information physically connected to the inclined road surface Pb, and is included in the captured image. When the coordinate of the inclined road surface Pb is converted to the flat road surface Pa, the obstacle related to the obstacle three-dimensional arrangement information in the captured image is also coordinate-converted and connected to the flat road surface Pa to the inclined road surface. By configuring the vehicle to automatically travel on the road surface Pb, it is possible to automatically travel by accurately recognizing an obstacle physically connected to the inclined road surface Pb.

  In this case, the obstacle three-dimensional arrangement information includes at least one of curb information physically connected to the slope road surface Pb, wall information, and roof information above a predetermined distance from the slope road surface Pb. Information is to be included.

[Modifications of Other Embodiments]
Next, as a modification of the example of FIG. 18 (example of traveling switching from a flat road surface to an inclined road surface), as shown in FIG. 21, for example, on a plane Pa of a non-contact charging area on a horizontal surface such as a smart home (not shown). Automatic parking (parking support) when a non-contact charging ground pad 210 (see also FIG. 19), which is a stepped object, is installed and a road surface marker 212 is drawn on the upper surface of the non-contact charging ground pad 210 This will be described as an example.

  The contactless ground pad 210 includes, for example, a primary ring-shaped power feeding coil connected to an AC power source (not shown). In this case, for example, a secondary ring-shaped power receiving coil 216 is disposed between the rear wheels 24 under the rear seat and the trunk of the vehicle 100a.

  By moving the vehicle 100a backward from the position of FIG. 21 in the direction of the arrow, a road surface marker 212 on which a primary side feeding coil (primary coil) is drawn (in a plan view, a circular frame, a rectangular frame, or a square frame) ) And the vehicle-mounted position of the secondary power receiving coil 216 (secondary coil) stored in the vehicle 100a are positioned on the same axis (vertical axis) in plan view, in other words For example, after the power feeding coil and the power receiving coil are positioned concentrically in plan view, the power receiving coil 216 on the secondary side of the vehicle 100a from the power feeding coil on the primary side of the ground pad 210 for non-contact charging. In contrast, by supplying electric power by electromagnetic induction or the like, it is possible to transmit electric power from the power supply coil side of the contactless ground pad 210 to the power reception coil 216 side of the vehicle 100a with high efficiency. In this manner, on the vehicle 100a side, the power supplied to the supplied power receiving coil 216 can be charged into a storage battery (not shown).

  In this case, as can be seen from FIG. 22, the correct position on the y coordinate (axis) yra along the road surface Pa of the road surface marker Ma (referred to as a point) drawn on the upper surface (front surface) of the ground pad 210 for non-contact charging. The coordinates are the coordinate position yra (yi1, Pc), and the corresponding correct position coordinate on which the road surface marker Ma on the axis yrc along the road surface marker 212 is drawn is the position of the y coordinate yrc (yi1, Pc). However, since there is a thickness ΔH ′ of the non-contact charging ground pad 210, the position of the road surface marker Ma is misidentified as the coordinate position yra (yi1, Pa). In other words, the correct length from the origin yra = 0 on the axis yra along the road surface Pa is yra (yi1, Pc), but it is erroneously recognized as a longer length yra (yi1, Pa). Thus, a distance error E occurs.

  In the example of FIG. 22, since the pad surface of the contactless ground pad 210 on which the road surface marker Ma is drawn is on the surface Pc parallel to the road surface Pa, the tilt angle Δθ (see FIG. 17) is Δθ = 0 value. become.

In this case, the angle β formed by the line Lma (including the extension line) connecting the focal point f and the position of the coordinate (distance) yi1 on the imaging surface 20i and the optical axis La is the same as the above formula (1). The following equation (8) is obtained.
β = arctan (yi1 / F) (8)

The y coordinate position yrc (yi1, Pc) of the road surface marker M on the y coordinate (axis) yrc is obtained by the following equation (9).
yrc (yi1, Pc) = (H−ΔH ′) × tan (θ + β) (9)

Since the tilt angle Δθ is Δθ = 0, yrc (yi1, Pc) and yra (yi1, Pc) are equivalent. Therefore, yra (yi1, Pc) can be obtained by the following equation (10).
yra (yi1, Pc) = yrc (yi1, Pc) (10)

  Thus, in [Modification of Other Embodiments], for example, a non-contact charging ground pad 210 that is a stepped object is installed on a flat surface Pa of a non-contact charging area on a horizontal surface of a smart home (not shown). Even when the road surface marker 212 (road surface marker Ma) is drawn on the upper surface of the contactless ground pad 210, automatic parking (parking assistance) can be accurately performed.

  In this case, the parking support system 14 (the infrastructure camera 106 is not required) includes the vehicle-mounted cameras 10 and 20 and the automatic driving unit 57, and the vehicle 100a having the own vehicle position detection function uses the vehicle-mounted cameras 10 and 20 to check the vehicle surrounding situation. In the parking support system 14 (infrastructure camera 106 is unnecessary) that automatically travels within the parking facility 12 (including a parking facility such as a parking lot attached to a home such as a smart house) while acquiring the vehicle 100a, When the vehicle automatically travels, the automatic driving unit 57 detects the road surface position information and the road surface level difference between the non-contact charging ground pad 210 and the road surface marker (marker indicating the level difference object) 212 which are the level difference objects in the parking facility 12. Information (thickness ΔH ′) is acquired from map information storage means (on-vehicle storage unit or storage unit such as an external server), and the vehicle 10 When it is determined that the travel position of 0a has reached the position of the contactless ground pad 210, the contactless charging site in the captured image acquired by the in-vehicle cameras 10 and 20 based on the road surface step information (thickness ΔH ′). The road surface marker 212 on the upper pad 210 is regarded as being on the traveling road surface (plane Pa) immediately below the surface Pc by the thickness ΔH ′, and then coordinate conversion (viewpoint conversion, reverse projection conversion) is performed without contact. It is possible to automatically park the power receiving coil 216 (non-contact power receiving unit) of the vehicle 100a on the ground pad for charging 210 (non-contact power feeding unit) in an accurate positional relationship, and to charge the on-vehicle storage battery with high efficiency. it can.

  Note that the present invention is not limited to the above-described embodiment and other embodiments, and it is needless to say that various configurations can be adopted based on the description in this specification.

12 ... Parking facility 14 ... Parking support system 50 ... Sensor 52 ... Overall ECU
56 ... Automatic driving ECU 57 ... Automatic driving units 100, 100 ', 100a ... Vehicles 102, 102A, 102B ... Storage positions 106, 106a-106d ... Infrastructure cameras 108, 108a-108d, 110, 110a-110d ... Captured images 120 ... Parking support control devices 121-126, 121a-126a ... parking positions 130, 132, 134, 136 ... movement path 210 ... non-contact charging ground pad

Claims (4)

In a parking support system in which a vehicle having an in-vehicle camera and an automatic driving unit and having a vehicle position detection function acquires a vehicle surrounding situation by the in-vehicle camera, and automatically travels in a parking facility
When the vehicle automatically travels within the parking facility, the automatic driving unit acquires the information on the sloped road surface in the parking facility from the terminal on the parking facility side, and the traveling position of the vehicle has reached the sloped road surface. When the vehicle is determined to be, the parking assist system is characterized in that the inclined road surface in the captured image acquired by the in-vehicle camera is coordinate-converted into a flat road surface based on the inclined road surface information and automatically travels on the inclined road surface. In the parking assistance system according to claim 1,
The inclined road surface information includes position information where the inclined road surface exists and gradient information of the inclined road surface. In the parking assistance system according to claim 1 or 2,
The inclined road surface information includes three-dimensional arrangement information of obstacles physically connected to the inclined road surface, and when the coordinate of the inclined road surface in the captured image is converted to the flat road surface, the photographing is also performed. A parking support system, wherein an obstacle related to the three-dimensional arrangement information of the obstacle in the image is coordinate-transformed and connected to the flat road surface to automatically travel on the inclined road surface. In the parking assistance system according to claim 3,
The obstacle three-dimensional arrangement information includes at least one of curb information physically connected to the inclined road surface, wall information, and roof information on the inclined road surface. system.

Images