JP2020192905A - Parking support device and parking support method - Google Patents

Abstract

To obtain a parking support device for preventing the impossibility of mark detection from having influence on vehicle parking support.SOLUTION: A camera unit 205 photographs a landmark arranged to define a parking section, a landmark position detection unit 208 recognizes the landmark to detect the position of the landmark from a camera video image of the landmark on the basis of landmark map information including the position of the landmark stored in advance in a landmark position storage unit 207, a landmark recognition possibility determination unit 206 determines whether the landmark can be recognized from the camera video image by the landmark position detection unit 208 when a parking support control unit 211 performs parking support to a parking section of the vehicle on the basis of the detected position of the landmark, and a notification unit 210 notifies a driver or a third person whether the landmark can be recognized.SELECTED DRAWING: Figure 3

Description

The present application relates to a parking support device and a parking support method for assisting parking of a vehicle in a parking lot.

The following are known as conventional parking support devices (for example, Patent Document 1).
The marker that defines the parking lot is recognized by the sensor mounted on the vehicle, and the positional relationship between the parking lot and the vehicle is specified from the position information of the marker held in advance and the position information of the marker obtained from the sensor, and the vehicle. It is stated that the vehicle is guided to the target position of the parking lot.

Japanese Patent No. 3804161 (pages 6-9, FIG. 1)

It is possible that the conventional parking support device as described above cannot recognize the marker for some reason, and in that case, it is not possible to guide the entry and exit of the vehicle.
For example, if the marker is a landmark with an optical pattern and the sensor is a camera that captures the environment around the vehicle, if the landmark is dirty, or if it blocks the light rays between the landmark and the camera. If an obstacle is placed, it is assumed that the landmark cannot be clearly photographed by the camera and the landmark cannot be recognized.

A third party (hereinafter referred to as a parking manager) such as a vehicle driver or a parking lot manager who is trying to receive parking support knows that the marker cannot be recognized by the conventional parking support device. This is because the parking manager cannot eliminate the marker recognition obstacle in advance because it is after requesting the entry / exit support.
That is, there is a problem that the vehicle cannot be quickly entered and exited when the marker cannot be recognized.

The present application discloses a technique for solving the above-mentioned problems, and an object of the present application is to provide a parking support device and a parking support method for preventing the influence of the undetectable marker on the parking support of a vehicle. ..

The parking support device disclosed in the present application is a parking support device that assists the parking of a vehicle in a parking section, and includes a marker detection unit that detects a marker arranged to define the parking section, and a marker position. A marker detection possibility determination unit that determines whether or not the marker can be detected based on the marker position storage unit stored in advance, the marker detection status by the marker detection unit, and the marker map information. It is provided with a notification unit for notifying the determination result of the detection / rejection determination unit.

According to the parking support device disclosed in the present application, it is possible to prevent the influence of the undetectable marker on the parking support of the vehicle.

It is a figure which shows the outline of the parking support device by Embodiment 1. FIG. It is explanatory drawing explaining the operation of the parking support apparatus according to Embodiment 1. FIG. It is a block diagram which shows the schematic structure of the parking support device according to Embodiment 1. It is a flowchart which shows the operation of the parking support device by Embodiment 1. It is a flowchart which shows the operation of the landmark monitoring mode of the parking support device by Embodiment 1. FIG. It is a flowchart which shows the operation of the warehousing / delivery mode of the parking support device by Embodiment 1. FIG. It is a figure which shows the example of the landmark recognition admissibility determination result of the parking support device by Embodiment 1. FIG. It is a figure which shows the position calculation method of the vehicle in the parking support device by Embodiment 1. FIG. It is a figure which shows the hardware composition of the parking support device by Embodiment 1. FIG.

Embodiment 1.
In the first embodiment, a parking support device and a parking support method for assisting the entry or exit of a vehicle in a parking lot in which a marker defining a parking lot is arranged will be described.
FIG. 1 is a diagram showing an outline of a parking support device according to the first embodiment.
In FIG. 1, the vehicle 101 is a vehicle that is a target of parking support (support for entering or leaving a vehicle). The wide-angle camera 102 is arranged in front of and behind the vehicle 101. Landmark 103 is a marker placed on the ground to define a parking lot and has an optical pattern. The landmarks 103a, 103b, 103c and 103d indicate individual landmarks 103. Parking lot 107 is a lot defined by landmark 103.
The number and location of the wide-angle cameras 102 and the number and position of the landmarks 103 are not limited to this.

FIG. 2 is an explanatory diagram illustrating the operation of the parking support device according to the first embodiment.
In FIG. 2, reference numerals 101 to 103 and 107 are the same as those in FIG. The dirt 104 is attached to the landmark 103. The obstacle 105 is an obstacle that hinders the visibility of the landmark 103.

FIG. 3 is a block diagram showing a schematic configuration of the parking support device according to the first embodiment.
In FIG. 3, the vehicle state detection unit 201 detects vehicle information including information on the IG (ignition) power supply. The operation unit 202 is for the driver or the administrator to input. The time detection unit 203 detects the time information.
The parking support system management unit 204 extracts information on the IG power supply from the vehicle information, and determines whether the driver is driving, the driver gets off, and the vehicle is parked.
The camera unit 205 (marker detection unit) has a wide-angle camera 102 arranged in the vehicle 101, and photographs a landmark 103 provided for defining a parking lot.

The landmark position detection unit 208 (marker position calculation unit) recognizes the landmark from the information obtained from the camera unit 205 and calculates the position information. The own vehicle position detection unit 209 (vehicle position calculation unit) calculates the relative position between the parking section 107 and the vehicle 101 to be parked.
The landmark recognition possibility determination unit 206 (marker detection possibility determination unit) monitors the landmark 103 even when the vehicle is not under the supervision of the driver or a third party, and determines whether the landmark 103 can be recognized. Therefore, the landmark recognition availability determination unit 206 determines whether or not the landmark in the camera image can be recognized based on the camera image and the landmark map information of the landmark position storage unit 207 described later.
The landmark position storage unit 207 (marker position storage unit) stores landmark map information (marker map information) including the landmark position.

The notification unit 210 notifies the driver of the vehicle or a third party of the determination result of the landmark recognition / rejection determination unit 206.
The parking support control unit 211 supports the vehicle 101 to enter or leave the parking lot 107 based on the positional relationship between the parking lot 107 and the vehicle 101. The parking support control unit 211 performs parking support control based on the vehicle position information, the vehicle information, and the parking support system command.

FIG. 7 is a diagram showing an example of a landmark recognition / rejection determination result of the parking support device according to the first embodiment.
In FIG. 7, the shooting time, the landmark ID, the recognizability, and the shot image are shown. FIG. 7 shows whether or not four landmarks can be recognized for each of the cases where the shooting times are 11:00 and 12:00.

FIG. 8 is a diagram showing a method of calculating the position of the vehicle in the parking support device according to the first embodiment.
FIG. 8A is a diagram showing landmark map information output by the landmark position storage unit 207, and FIG. 8B is a diagram and a diagram showing the landmark position detection result output by the landmark position detection unit 208. 8 (c) is a diagram showing a state in which the landmark map information and the landmark position detection result are matched by the own vehicle position detection unit 209.
In FIG. 8, reference numeral 101 is the same as that in FIG. The landmark position information A, B, C, and D are landmark positions stored in the landmark map information, and the landmark position information a, b, c, and d are detected by the landmark position detection unit 208. It is a landmark position.

The landmark map information shown in FIG. 8 (a) was accurately surveyed and recorded when the landmark was laid. The respective landmark position information A, B, C, and D are position information from the coordinate origin O of the map, and are values (true values) that do not include an error.
The landmark position detection result of FIG. 8B is continuously extracted during parking control based on the camera image, and the calculated landmark position information is from the coordinate origin O'of the vehicle 101. The landmark position information a, b, c, and d are values (measured values) including errors.
Both landmark position information has a correspondence relationship (Aa, Bb, Cc, Dd).

Next, the operation will be described.
The operation of the parking support device will be described with reference to FIG.
First, in step S301, the vehicle state detection unit 201 detects vehicle information. Next, in step S302, the parking support system management unit 204 extracts information on the IG power supply from the vehicle information, and determines whether the driver is driving, the driver gets off, and the vehicle is parked.

In the first embodiment, whether or not the driver is driving is determined by the vehicle's IG power supply signal (when the IG power supply voltage is applied, the driver is driving and is not applied, off. In the case of, it is judged by parking), but using a method other than IG power supply (for example, vehicle information such as shift position information, driver seat seating sensor information, keyless entry system or immobilizer system information) Then, it may be determined whether the driver is driving the vehicle or is parked and absent.

When the driver gets off in step S302 and it is determined that the vehicle has been parked, the parking support system management unit 204 parks the vehicle in step S303 at the time of the immediately preceding parking operation using the main parking support device. Determine if you have done so.
When parking is performed using the main parking support device at the time of the previous parking operation, it is determined that the vehicle has already been parked in the parking section where the landmark is arranged, and the landmark monitoring mode M1 is operated.
The parking support system management unit 204 has a storage unit (nonvolatile memory, etc.) for storing the previous operating state.

On the other hand, when it is determined in step S302 that the driver is driving, the parking support system management unit 204 acquires the driver's parking support request at the operation unit 202 in step S304. Then, in step S305, when the parking support system management unit 204 determines that there is a driver's parking support request, the warehousing / delivery mode M2 is operated.

Next, the details of the operation of the landmark monitoring mode M1 will be described with reference to FIG.
The landmark monitoring mode M1 is a mode in which the driver repeatedly operates when the driver gets off and the vehicle is parked. As long as the condition of the landmark monitoring mode M1 is satisfied, the processing of the loop L1 is always executed.
Loop L1 has the processing of steps S311 to S319.

The processing of step S311 and step S312 is the same as the processing of step S301 and step S302, the vehicle state detection unit 201 detects the vehicle information, and the parking support system management unit 204 obtains the IG power supply information from the vehicle information. Take it out and determine if the driver is driving or if the driver gets off and the vehicle is parked.

The determination in step S312 is a determination as to whether or not to exit the landmark monitoring mode M1. When the vehicle changes from the parked state to the driving state of the river, the loop L1 is escaped and the landmark is determined. The monitoring mode M1 is terminated.
As described above, in the first embodiment, whether or not the driver is driving is determined by the IG power supply signal of the vehicle, but the driver is driving the vehicle by using a method other than the IG power supply. It may be determined whether the vehicle is parked or absent.

Next, in step S313, the time information is detected by the time detection unit 203.
Subsequently, in step S314, the parking support system management unit 204 determines whether or not the landmark monitoring time is the time when a certain time has elapsed from the previous operation time.
Further, the parking support system management unit 204 extracts information on the battery power supply from the vehicle information detected by the vehicle state detection unit 201, and determines whether or not the battery voltage is sufficient.

In step S314, the parking support system management unit 204 performs the processes of steps S315 to S319 only when the battery voltage is finally sufficient and the landmark monitoring time, and if different, the end point of the loop L1. Proceed to.
In step S314, when the parking support system management unit 204 determines that the battery voltage is sufficient and the landmark monitoring time is reached, in step S315, the parking support system management unit 204 issues "landmark monitoring" as a parking support system command. "Mode" is output.

Next, in step S316 (first step), when the camera unit 205 receives the "landmark monitoring mode" as a parking support system command, the camera unit 205 captures an image around the vehicle and outputs the camera image.

Next, in step S317 (second step), the landmark recognition availability determination unit 206 determines whether or not the landmark in the camera image can be recognized based on the camera image and the landmark map information of the landmark position storage unit 207. judge.
The landmark position storage unit 207 stores the number of landmarks to be detected and the position of each landmark, and the landmark recognition possibility determination unit 206 stores a predetermined position (for example, landmark position storage) in the camera image. It is determined for each landmark whether or not the landmark is detected (recognized) in the range of ± 30 cm of the landmark position stored in the unit 207.
The landmark recognition / rejection determination unit 206 outputs the number of landmarks determined to be recognizable, the number of landmarks determined to be unrecognizable, and the image around the landmarks determined to be unrecognizable.

For example, in FIG. 2, of the four landmarks 103a to 103d) to be detected, the landmarks 103a and 103d cannot be detected due to dirt 104 or obstacle 105, respectively, so that the determination as shown in FIG. 7 is made. Output the result.

Various methods can be considered for the determination method of the landmark recognition / rejection determination unit 206, but when the landmark has a simple shape, for example, a straight line or a circle is detected by using a Hough transform or the like. The landmark may be determined by the combination thereof and the size / length. When the landmark has a complicated shape, a method such as feature point matching or template matching may be used.

Further, in the first embodiment, a landmark having an optical pattern is used as a marker, and the landmark is detected by using a wide-angle camera. However, since the position of the own vehicle with respect to the parking lot can be specified. If there is, other combinations of markers and sensors (for example, a combination of a light emitting element and a camera, a combination of an infrared light emitting element and an infrared sensor, etc.) may be used.

Next, in step S318, the parking support system management unit 204 receives the determination result of the landmark recognition / rejection determination unit 206, and if there is one or more landmarks determined to be unrecognizable, the number and recognition of the landmarks. The image of the landmark determined to be impossible is output to the notification unit 210.
In the first embodiment, when the number of landmarks determined to be unrecognizable is one or more, it is output to the notification unit 210, but other conditions (for example, all landmarks). If 50% or less of the number is determined to be undetectable, if the number of landmarks determined to be detectable is less than 3, etc.), it may be determined whether or not to output to the notification unit 210. ..

If the number of landmarks determined to be unrecognizable in step S318 is less than one, the loop L1 is escaped.

When the number of landmarks determined to be unrecognizable in step S318 is one or more, in step S319 (third step), the notification unit 210 sends a message to the effect that there are landmarks that cannot be recognized. , Notify the manager of the vehicle 101.

The method and configuration for notifying the administrator of the message is not limited to the first embodiment, but for example, the driver is notified using the screen of the navigation device or the information screen of the navigation device equipped on the vehicle 101. You may.
Further, since it is possible that there is no driver in the vehicle 101, the driver's mobile terminal (smartphone or the like) may be notified.
Further, in the case of a parking management mode in which the parking lot manager collectively manages a plurality of parked vehicles, the parking lot manager may notify the parking lot control device instead of the driver of the vehicle 101.
Furthermore, third parties involved in vehicle management or parking management, such as parking guards, may be notified.

The content of the notification in these cases may be simply a notification that one or more landmarks could not be detected, or more detailed information as shown in FIG. 7 (the number of landmarks that could not be detected). , Which landmark could not be detected, an image taken around the landmark that could not be detected, etc.) may be notified.

In order to improve convenience, the timing at which the notification unit 210 notifies the administrator may be arbitrarily set by the administrator. In that case, the parking support system management unit 204 has a setting storage unit for storing the set value of the administrator and a result storage unit (both are non-volatile memories, etc.) for storing the result of the landmark recognition / rejection determination so far. Shall have.
In advance, the administrator saves the setting of the notification timing such as the notification time in the setting storage unit of the parking support system management unit 204 through the operation unit 202. Then, each time the parking support system management unit 204 monitors the landmark, it saves the recognition / rejection determination result as shown in FIG. 7 in the result storage unit, and when the timing is saved in the setting storage unit, the latest recognition The pass / fail judgment result and the like are output to the notification unit 210.

Next, the details of the operation of the warehousing / delivery mode M2 will be described with reference to FIG.
The warehousing / delivery mode M2 is a mode that operates when the driver puts the vehicle 101 in the parking lot 107 or leaves the parking lot 107, and performs the processes of steps S321 to S326.

In step S321, similarly to step S301, the vehicle state detection unit 201 detects the vehicle information, and in step S322, the parking support system management unit 204 outputs the "warehousing / delivery mode" as the parking support system command.

Next, in step S323, when the camera unit 205 receives the "entry / exit mode" as the parking support system command, it captures an image of the surroundings of the vehicle and outputs the camera image.

Next, in step S324 (fourth step), the landmark position detection unit 208 detects the landmark position in the camera image based on the camera image and the landmark map information of the landmark position storage unit 207. ..

Next, in step S325 (fifth step), the own vehicle position detection unit 209 calculates the relative position between the vehicle 101 and the parking lot 107.

Here, a method of calculating the position of the vehicle 101 in the own vehicle position detection unit 209 will be described with reference to FIG.
The vehicle position detection unit 209 has the landmark map information stored in the landmark position storage unit 207 shown in FIG. 8A and the landmark position detection unit 208 shown in FIG. 8B. Both of the landmark position detection results to be output are input.

Since the landmark position detection unit 208 extracts the landmark position detection result based on the landmark map information, in the own vehicle position detection unit 209, both landmark position information are already in a corresponding relationship (Aa, Bb, Cc, Dd) are clarified.
The own vehicle position detection unit 209 performs matching processing using the correspondence between the landmark map information and the landmark position detection result.

The matching process performs translation and rotation on the landmark position detection result, and calculates using the least squares method so that the position deviation from the landmark map information is minimized.
As described above, since the landmark position detection result includes an error at the time of detection, even if the landmark position detection result is translated and rotated, it does not match the landmark map information. By using the method of least squares, the distance and angle of rotation of the translation that minimizes the error can be obtained.
The translation distance and rotation angle of the landmark position detection result obtained by the own vehicle position detection unit 209 are the position and angle of the vehicle coordinate origin O'based on the map coordinate origin O, that is, the vehicle position. It becomes an angle.
The own vehicle position detection unit 209 outputs the position and angle of the vehicle as the own vehicle position information with respect to the parking lot 107.

Next, in step S326 (sixth step), the parking support control unit 211 performs parking support control based on the own vehicle position information, the vehicle information, and the parking support system command.
The parking support method may be the same as the conventional parking support method as shown in Japanese Patent No. 6257718, for example.

As described above, in the parking support device of the first embodiment, the vehicle is placed in the parking lot based on the landmark monitoring mode in which the vehicle is periodically operated for the purpose of confirming whether or not the landmark can be recognized while the vehicle is parked, and the driver's will. It has an entry / exit mode for the purpose of parking or leaving the parking lot.
Then, the parking lot manager is notified of the determination result of whether or not the landmark can be recognized in the landmark monitoring mode, if necessary, before the execution of the warehousing / delivery mode.

According to the first embodiment, the parking manager can eliminate the obstacle of the landmark by notifying the determination result of whether or not the landmark can be recognized.
Therefore, it is possible to prevent the vehicle from being unable to enter / exit due to undetected landmarks when the entry / exit mode is executed, that is, when the vehicle is entered / exited.

The parking support device 10 is composed of a processor 11 and a storage device 12 as shown in FIG. 9 as an example of hardware. Although the storage device is not shown, it includes a volatile storage device such as a random access memory and a non-volatile auxiliary storage device such as a flash memory. Further, an auxiliary storage device of a hard disk may be provided instead of the flash memory. The processor 11 executes the program input from the storage device 12. In this case, the program is input from the auxiliary storage device to the processor 11 via the volatile storage device. Further, the processor 11 may output data such as a calculation result to the volatile storage device of the storage device 12, or may store the data in the auxiliary storage device via the volatile storage device.

Although the present disclosure describes exemplary embodiments, the various features, aspects, and functions described in the embodiments are not limited to the application of a particular embodiment, but alone. Alternatively, it can be applied to embodiments in various combinations.
Therefore, innumerable variations not illustrated are envisioned within the scope of the techniques disclosed herein. For example, it is assumed that at least one component is modified, added or omitted.

10 parking assistance device, 11 processor, 12 storage device, 101 vehicle,
102 wide-angle camera, 103 landmarks, 104 dirt, 105 obstacles,
107 Parking lot, 201 Vehicle condition detection unit, 202 Operation unit, 203 Time detection unit,
204 Parking Support System Management Department, 205 Camera Department,
206 landmark recognition / rejection determination unit, 207 landmark position storage unit,
208 Landmark position detection unit, 209 Vehicle position detection unit, 210 Notification unit,
211 Parking support control unit

The parking support device disclosed in the present application is a parking support device that assists the parking of a vehicle in a parking lot, and includes a marker detection unit that detects a marker arranged to define the parking lot, and a marker position. The marker detection possibility determination unit that determines whether or not the marker can be detected based on the marker position storage unit stored in advance, the marker detection status by the marker detection unit, and the marker map information, this marker. It is equipped with a notification unit that notifies the determination result of the detection / rejection determination unit and a parking determination unit that determines whether the vehicle is driving or parked in the parking zone . When the parking determination unit determines that the vehicle is parked in the parking zone. , The marker detection unit, the marker detection availability determination unit, and the notification unit operate .

FIG. 3 is a block diagram showing a schematic configuration of the parking support device according to the first embodiment.
In FIG. 3, the vehicle state detection unit 201 detects vehicle information including information on the IG (ignition) power supply. The operation unit 202 is for the driver or the administrator to input. The time detection unit 203 detects the time information.
The parking support system management unit 204 (parking determination unit) extracts information on the IG power supply from the vehicle information, and determines whether the driver is driving, the driver gets off, and the vehicle is parked.
The camera unit 205 (marker detection unit) has a wide-angle camera 102 arranged in the vehicle 101, and photographs a landmark 103 provided for defining a parking lot.

Determination in step S312 is a judgment as to whether or not to end the landmark monitoring mode M1, the vehicle is, from the parking already state, if the drivers has changed state during operation, escaped loop L1, the land The mark monitoring mode M1 is terminated.
As described above, in the first embodiment, whether or not the driver is driving is determined by the IG power supply signal of the vehicle, but the driver is driving the vehicle by using a method other than the IG power supply. It may be determined whether the vehicle is parked or absent.

The parking support device disclosed in the present application is a parking support device that assists the parking of a vehicle in a parking lot, and is a marker detection unit that detects a marker mounted on the vehicle and arranged to define the parking lot. , Marker detection that determines whether or not the marker map information including the marker position is in a detectable state based on the marker detection status of the marker position storage unit and the marker detection unit stored in advance and the marker map information. It is equipped with a availability determination unit, a notification unit that notifies the determination result of this marker detection availability determination unit, and a parking determination unit that determines whether the vehicle is driving or parked in the parking zone. When it is determined, the marker detection unit, the marker detection feasibility determination unit, and the notification unit operate.

Claims (12)

A parking support device that assists the parking of vehicles in the parking lot.
A marker detector that detects a marker placed to define the parking lot,
The marker position storage unit, in which the marker map information including the position of the marker is stored in advance,
A marker detection possibility determination unit that determines whether or not the marker is in a detectable state based on the detection status of the marker by the marker detection unit and the marker map information.
A parking support device including a notification unit for notifying the determination result of the marker detection possibility determination unit. The parking support device according to claim 1, wherein the marker detection unit is mounted on the vehicle. The parking support device according to claim 1 or 2, wherein the marker detection unit is a camera. The parking support device according to any one of claims 1 to 3, wherein the marker detection possibility determination unit determines the state of the marker at a predetermined time. A marker position calculation unit that recognizes the marker from the detection result by the marker detection unit and calculates the position of the marker.
A vehicle position calculation unit that calculates a relative position between the vehicle and the parking lot based on the marker position calculated by the marker position calculation unit and the marker position in the marker map information.
Based on the relative position between the vehicle and the parking lot calculated by the vehicle position calculation unit, the parking support control unit for supporting the entry of the vehicle into the parking lot or the exit from the parking lot is provided. The parking support device according to any one of claims 1 to 4, which is characterized. The notification unit is characterized in that the parking support control unit notifies the determination result of the marker detection possibility determination unit before the parking support control unit supports the entry of the vehicle into the parking area or the exit from the parking area. The parking support device according to claim 5. The parking support device according to any one of claims 1 to 6, wherein the notification destination of the notification by the notification unit is a mobile terminal. The parking support device according to any one of claims 1 to 6, wherein the notification destination of the notification by the notification unit is a control device for a parking lot having the parking lot. The notification by the notification unit includes any one of claims 1 to 8, wherein the notification includes the number of markers determined to be in an undetectable state by the marker detection availability determination unit. The parking assistance device described. The parking support device according to claim 3, wherein the notification by the notification unit includes an image taken by the camera. It is a parking support method that supports the parking of vehicles in the parking lot.
The first step, in which the marker detector detects a marker arranged to define the parking lot,
The marker detection possibility determination unit determines whether or not the marker is in a detectable state based on the detection status of the marker by the first step and the marker map information including the position of the marker created in advance. Two steps,
A parking support method, wherein the notification unit includes a third step of notifying the determination result of the second step. The fourth step, in which the marker position calculation unit recognizes the marker detected by the marker detection unit and calculates the position of the marker.
The fifth step, in which the vehicle position calculation unit calculates the relative position between the vehicle and the parking lot based on the position of the marker calculated by the fourth step and the position of the marker in the marker map information.
Based on the relative position between the vehicle and the parking lot calculated in the fifth step, the parking support control unit supports the entry of the vehicle into the parking lot or the exit from the parking lot. The parking support method according to claim 11, wherein the parking support method includes steps.

Images