JP6828501B2 - Parking support device - Google Patents

Description

The present invention relates to a parking support device.

Parking support devices that support the parking of vehicles in parking lots and the like are known. The parking support device detects a lane marking indicating the parking area of one vehicle from the captured image to support parking. When such a parking support device detects a lane marking, a search region (also referred to as ROI (Region Of Interest)) is set in a part of the captured image, and the lane marking is detected in the search region. To do.

Japanese Unexamined Patent Publication No. 2006-175918

However, the above-mentioned parking support device has a problem that the search area cannot be appropriately set in the captured image depending on the moving direction of the vehicle operated by the driver.

The present invention has been made in view of the above, and provides a parking support device capable of setting a search area at an appropriate position in a captured image regardless of the moving direction of a vehicle or the like.

In order to solve the above-mentioned problems and achieve the object, the parking support device of the present invention provides a search area for searching a lane marking that divides the parking area in the captured image obtained by capturing the periphery of the moving vehicle. The setting unit includes a setting unit for setting and a detection unit for searching in the search area to detect the division line, and the setting unit sets the detected position of the division line and a preset setting interval . Based on this, the position of the search area for searching for another division line is set.

As described above, in the present invention, by setting the position of the next search area based on the detected lane markings, the lane markings that are periodically arranged are detected regardless of the moving direction of the vehicle or the like. Since the search area of can be set at an appropriate position in the captured image, the detection accuracy of the lane marking can be improved. Further, since the search area is set based on the preset setting interval, the processing load for setting the search area can be reduced.

In the parking support device of the present invention, the detection unit detects a plurality of lane markings, and the setting unit determines the position of the search area for searching the other lane markings based on the plurality of lane markings. It may be set.

As described above, since the present invention sets the position of the next search area based on the plurality of lane markings, the position of the search area for detecting the periodically arranged lane markings is set more appropriately. it can.

In the parking support device of the present invention, the setting unit searches for the other lane markings at a plurality of different positions based on the detected positions of the lane markings and a plurality of preset interval settings. The position of the search area may be set.

As described above, since the search area is set based on the preset setting interval in the present invention, the load of processing for setting the search area is reduced, and even in parking lots having different lane marking intervals. The position of the search area can be set appropriately.

In the parking support device of the present invention, the setting unit sets the search area only on the right side of the vehicle when the vehicle is turning left, and sets the search area on the vehicle when the vehicle is turning right. It may be set only on the left side of.

As described above, according to the present invention, by setting the search area on only one of the left and right sides of the vehicle depending on the traveling direction of the vehicle, the burden of processing for setting the search area can be reduced.

FIG. 1 is a plan view of the vehicle of the embodiment. FIG. 2 is a block diagram illustrating a configuration of a parking support system mounted on a vehicle. FIG. 3 is a functional block diagram illustrating the function of the parking support device. FIG. 4 is a plan view illustrating the setting of the search area in the real world until a plurality of lane markings are detected. FIG. 5 is a plan view illustrating the setting of the search area according to the first embodiment in the real world after detecting a plurality of lane markings. FIG. 6 is a plan view illustrating the movement of the vehicle. FIG. 7 is a plan view illustrating a coordinate system in the real world with the center of the rear wheel axle of the vehicle as the origin. FIG. 8 is a flowchart of the parking support process according to the first embodiment. FIG. 9 is a plan view illustrating the setting of the search area according to the second embodiment in the real world after the lane marking is detected. FIG. 10 is a flowchart of the parking support process according to the second embodiment. FIG. 11 is a plan view illustrating the setting of the search area according to the third embodiment in the real world after the lane marking is detected. FIG. 12 is a flowchart of the left / right determination process of the fourth embodiment. FIG. 13 is a plan view illustrating the setting of the search area in the real world until the setting unit of the fifth embodiment detects a plurality of division lines. FIG. 14 is a plan view illustrating the setting of the search area in the real world until the setting unit of the sixth embodiment detects a plurality of division lines. FIG. 15 is a plan view illustrating the setting of the search area in the real world until the setting unit of the seventh embodiment detects a plurality of division lines.

Similar components such as the following exemplary embodiments are given common reference numerals, and duplicate description will be omitted as appropriate.

<First Embodiment>
FIG. 1 is a plan view of the vehicle 10 of the first embodiment. As shown in FIG. 1, the vehicle 10 has a vehicle body 12 and four imaging units 14a, 14b, 14c, and 14d. When it is not necessary to distinguish the imaging units 14a, 14b, 14c, and 14d, it is referred to as the imaging unit 14.

The vehicle body 12 constitutes a passenger compartment in which an occupant rides. The vehicle body 12 accommodates or holds the configuration of the vehicle 10 such as the wheels 13 and the imaging unit 14.

The image pickup unit 14 is, for example, a digital camera incorporating an image pickup element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging unit 14 outputs moving image or still image data including a plurality of frame images generated at a predetermined frame rate as captured image data. The imaging unit 14 has a wide-angle lens or a fisheye lens, respectively, and can photograph a range of 140 ° to 190 ° in the horizontal direction. The optical axis of the imaging unit 14 is set obliquely downward. Therefore, the image pickup unit 14 outputs the data of the captured image obtained by capturing the periphery of the vehicle 10 including the surrounding road surface.

The imaging unit 14 is provided around the vehicle body 12. For example, the image pickup unit 14a is provided at a central portion (for example, a front bumper) in the left-right direction of the front end portion of the vehicle body 12. The imaging unit 14a generates an captured image of the periphery in front of the vehicle 10. The image pickup unit 14b is provided at a central portion (for example, a rear bumper) in the left-right direction of the rear end portion of the vehicle body 12. The imaging unit 14b generates an captured image of the rear periphery of the vehicle 10. The imaging unit 14c is provided at the center of the left end of the vehicle body 12 in the front-rear direction (for example, the left side mirror 12a). The imaging unit 14c generates an captured image of the left periphery of the vehicle 10. The imaging unit 14d is provided at the center of the right end of the vehicle body 12 in the front-rear direction (for example, the right side mirror 12b). The imaging unit 14d generates an image captured by capturing the right periphery of the vehicle 10.

FIG. 2 is a block diagram illustrating the configuration of the parking support system 20 mounted on the vehicle 10. As shown in FIG. 2, the parking support system 20 includes a wheel speed sensor 22, a steering angle sensor 24, an automatic driving unit 32, a monitoring device 34, a parking support device 36, and an in-vehicle network 38.

The wheel speed sensor 22 includes, for example, a Hall element provided in the vicinity of the wheel 13. The wheel speed sensor 22 outputs the number of wheel speed pulses indicating the detected rotation speed of the wheel 13 as a sensor value to the in-vehicle network 38.

The steering angle sensor 24 is provided on a steering portion such as a steering wheel or a steering wheel. The steering angle sensor 24 detects the amount of rotation of the steering unit or the steering angle, which is the rotation angle, operated by the driver. The steering angle sensor 24 is, for example, a rotary encoder or the like. The steering angle sensor 24 outputs the detected steering angle to the in-vehicle network 38.

The automatic driving unit 32 controls the accelerator, the brake, the steering wheel, and the like based on the instruction from the parking support device 36, and automatically drives the vehicle 10 at the time of parking.

The monitoring device 34 is provided on a dashboard or the like in the vehicle interior. The monitoring device 34 includes a display unit 40, an audio output unit 42, and an operation input unit 44.

The display unit 40 displays an image based on the image data transmitted by the parking support device 36. The display unit 40 is, for example, a display device such as a liquid crystal display (LCD) or an organic EL display (OELD). The display unit 40 displays, for example, an captured image in which a parking candidate area, which is a candidate for a parking area for the vehicle 10 in automatic driving, is superimposed.

The voice output unit 42 outputs voice based on the voice data transmitted by the parking support device 36. The audio output unit 42 is, for example, a speaker. The audio output unit 42 may be provided at a position in the vehicle interior different from that of the display unit 40.

The operation input unit 44 receives the input of the occupant. The operation input unit 44 is, for example, a touch panel. The operation input unit 44 is provided on the display screen of the display unit 40. The operation input unit 44 is configured to be transparent to the image displayed by the display unit 40. As a result, the operation input unit 44 can make the occupant visually recognize the image displayed on the display screen of the display unit 40. The operation input unit 44 receives the instruction input by the occupant touching the position corresponding to the image displayed on the display screen of the display unit 40, and transmits the instruction to the parking support device 36. For example, the operation input unit 44 outputs a parking instruction to the parking support device 36 when the occupant touches and selects the position of the parking candidate area displayed by the display unit 40.

The parking support device 36 is a computer including a microcomputer such as an ECU (Electronic Control Unit). The parking support device 36 acquires the data of the captured image from the imaging unit 14. The parking support device 36 transmits data related to an image or sound generated based on the captured image or the like to the monitor device 34. The parking support device 36 controls the automatic driving unit 32 to automatically drive and support the vehicle 10 when parking. The parking support device 36 includes a CPU (Central Processing Unit) 36a, a ROM (Read Only Memory) 36b, a RAM (Random Access Memory) 36c, a display control unit 36d, a voice control unit 36e, and an SSD (Solid State Drive). ) 36f. The CPU 36a, ROM 36b and RAM 36c may be integrated in the same package.

The CPU 36a is an example of a hardware processor, reads a program stored in a non-volatile storage device such as a ROM 36b, and executes various arithmetic processes and controls according to the program. The CPU 36a executes image processing such as an image for parking support to be displayed on the display unit 40, for example.

The ROM 36b stores each program and parameters required for executing the program. The RAM 36c temporarily stores various data used in the calculation by the CPU 36a. The display control unit 36d mainly executes image processing of the image obtained by the image pickup unit 14, data conversion of the image for display to be displayed on the display unit 40, and the like among the arithmetic processing in the parking support device 36. The voice control unit 36e mainly executes the voice processing to be output to the voice output unit 42 among the arithmetic processing in the parking support device 36. The SSD 36f is a rewritable non-volatile storage unit that maintains data even when the power of the parking support device 36 is turned off.

The in-vehicle network 38 is, for example, a CAN (Controller Area Network). The in-vehicle network 38 electrically connects the wheel speed sensor 22, the steering angle sensor 24, the automatic driving unit 32, the parking support device 36, and the operation input unit 44 so as to be able to transmit and receive signals and information to each other.

In the present embodiment, the parking support device 36 controls the parking support process of the vehicle 10 by the cooperation of hardware and software (control program). The parking support device 36 detects a lane marking that divides the parking area of the parking lot from the captured image including the peripheral image captured by the imaging unit 14, and sets one or a plurality of parking candidate areas. By controlling the automatic driving unit 32, the parking support device 36 moves the parking support device 36 into the parking candidate area selected by the occupant to support parking.

FIG. 3 is a functional block diagram illustrating the function of the parking support device 36. As shown in FIG. 3, the parking support device 36 has a control unit 50 and a storage unit 52.

The control unit 50 is realized as a function of, for example, the CPU 36a or the like. The control unit 50 includes a setting unit 54, a detection unit 56, and an operation control unit 58. The control unit 50 may realize the functions of the setting unit 54, the detection unit 56, and the operation control unit 58 by reading the parking support program 70 stored in the storage unit 52. A part or all of the setting unit 54, the detection unit 56, and the operation control unit 58 may be configured by hardware such as a circuit including an ASIC (Application Specific Integrated Circuit).

The setting unit 54 acquires an captured image of the periphery of the moving vehicle 10 from the imaging unit 14. For example, the setting unit 54 acquires a plurality of frame images of moving images that differ in time from the imaging unit 14 as captured images. The setting unit 54 sets a search area for searching for a division line that divides the parking area in the captured image, and outputs the search area to the detection unit 56. The setting unit 54 does not have to set the search area for all of the plurality of captured images (that is, the frame image). For example, the search area may be set every few frames, and the search set in advance may be set. The next search area may be set in the captured image acquired after the processing for the area is completed. The setting unit 54 sets a search area having a rectangular or parallelogram shape. An example of a lane marking is a white line provided on the ground of a parking lot. Here, the setting unit 54 takes an image based on one or a plurality of initial coordinates indicated by the initial coordinate data 72 preset and stored in the storage unit 52 until the detection unit 56, which will be described later, detects the division line. Set a search area for each in the image. Specifically, the setting unit 54 sets the relative position between the vehicle 10 and the search area as the initial coordinates in the real world until the lane marking is detected. Next, the setting unit 54 converts a plurality of captured images that change with the movement of the vehicle 10 by converting the coordinate system of the real world into the coordinate system of the captured image by a mapping table or the like for coordinate conversion set in advance. Set the search area for each of. In other words, the setting unit 54 sets the search area at the same position (that is, the same coordinates) in the captured image when the captured image is not enlarged or the like.

Further, when the setting unit 54 acquires the information of the detected division line from the detection unit 56, the setting unit 54 newly sets the position of the search area for searching another division line based on the detected division line. Even after the division line is detected, the setting unit 54 sets the position of the search area by setting the search area to a position in the real world and then converting the search area to a position in the captured image by a mapping table or the like. For example, the setting unit 54 sets a search area for searching for another division line based on the information of the plurality of division lines acquired from the detection unit 56. Specifically, the setting unit 54 calculates the detection interval, which is the interval between the division line and the adjacent division line. The setting unit 54 moves the center of the last detected division line in the front-rear direction by the detection interval along the direction perpendicular to the longitudinal direction of the last detected division line (hereinafter referred to as the perpendicular direction). It may be set as a new search area centered on the direction. Alternatively, the setting unit 54 may set a new search area at a position where the search area for which the lane marking was finally detected is moved along the perpendicular direction by the detection interval. When the setting unit 54 acquires the information of the plurality of lane markings from the detection unit 56, the setting unit 54 sets the parking candidate area between the adjacent lane markings and outputs the information to the operation control unit 58.

The detection unit 56 searches within the search area set in the captured image by the setting unit 54, and detects the lane marking. Further, the detection unit 56 searches within a plurality of search areas set in each of the plurality of captured images by the setting unit 54 to detect a plurality of division lines. The detection unit 56 outputs the information of the detected one or a plurality of division lines to the setting unit 54.

The driving control unit 58 controls the automatic driving unit 32 based on the parking candidate area set by the setting unit 54 and selected by the occupant and the lane marking detected by the detection unit 56 to automatically drive the vehicle 10. , The vehicle 10 is parked in the parking candidate area.

The storage unit 52 is realized as a function of the ROM 36b, the RAM 36c, the SSD 36f, and the like. The storage unit 52 stores the initial coordinate data 72 and the like indicating the coordinates for setting the search area in the execution of the parking support program 70 and the parking support program 70 executed by the control unit 50.

FIG. 4 is a plan view illustrating the setting of the search area SA in the real world until the plurality of division lines 90 are detected. When it is not necessary to distinguish each of the plurality of search areas SA1L, SA1R ..., It is referred to as the search area SA. When it is not necessary to distinguish between the plurality of positions CP1, CP2, ... Of the vehicle 10, it is referred to as the position CP.

FIG. 4 assumes a case where the vehicle 10 moves from the position CP1 through the positions CP2 and CP3 along the arrow to the position CP4 shown by the solid line. The position of the vehicle 10 is the center of the rear wheel axle of the vehicle 10, as indicated by the thick cross. Further, the position of the vehicle 10 is set as the origin, the left-right direction of the vehicle 10 is the X direction (the left direction is the + X direction), and the front-rear direction of the vehicle 10 is the Z direction (the rear is the + Z direction). Therefore, the origin and the XZ direction change with respect to the surrounding environment when the vehicle 10 changes the moving and traveling directions.

In this case, when the setting unit 54 acquires the captured image at the position CP1, the vehicle 10 is based on the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz) indicated by the preset initial coordinate data 72. Search areas SA1L and SA1R are set on both the left and right sides of. Specifically, the search region SA1L is set at the position of the left initial coordinates (+ Dx, Dz) with the position CP1 of the vehicle 10 as the origin. The search area SA1R is set at the position of the right initial coordinates (-Dx, Dz) with the position CP1 of the vehicle 10 as the origin. The left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz) may be the same except for the positive and negative signs of the X coordinates. Since there is no lane marking 90 in the search areas SA1L and SA1R, the detection unit 56 cannot detect the lane marking 90 at the position CP1.

When the setting unit 54 acquires a new captured image at the position CP2, the search areas SA2L and SA2R are set on the left and right sides of the vehicle 10 based on the left initial coordinates (+ Dx, Dz) and the right initial coordinates (-Dx, Dz). Set. Since there is no lane marking 90 in the search areas SA2L and SA2R, the detection unit 56 cannot detect the lane marking 90 at the position CP2. After that, the setting unit 54 repeats the processing at the position CP1 and the position CP2 until the detection unit 56 detects the division line 90, and sets the search area SA.

Therefore, the setting unit 54 executes the same processing as that of the position CP2 and the position CP1 at each of the position CP3 and the position CP4 to set the search areas SA3L, SA3R, SA4L, and SA4R. The detection unit 56 detects the lane marking 90 in the search areas SA3L, SA3R, SA4L, and SA4R. The detection unit 56 outputs information such as the position and direction of the detected division line 90 to the setting unit 54. When a plurality of lane markings 90 are detected on the same side of the left side or the right side of the vehicle 10, the setting unit 54 sets the left parking candidate area PAL and the right parking candidate area, which are candidates for the parking area of the vehicle 10 in automatic driving. Set the PAR. When it is not necessary to distinguish between the parking candidate area PAL on the left side and the parking candidate area PAR on the right side, it is described as the parking candidate area PA.

FIG. 5 is a plan view illustrating the setting of the search region SA according to the first embodiment in the real world after detecting the plurality of division lines 90. As shown in FIG. 5, when the setting unit 54 acquires information about the plurality of division lines 90 from the detection unit 56 and acquires a new captured image, the setting unit 54 obtains another division line 90 based on the plurality of division lines 90. Set the position of the search area SA to search for.

Specifically, the setting unit 54 is located between the lane markings 90 detected on the same left and right sides of the lane markings 90 detected in the search areas SA3L, SA3R, SA4L, and SA4R and the adjacent lane markings 90. The detection interval Dd, which is the interval of, is calculated. Next, the setting unit 54 detects the detection interval in the perpendicular direction (here, −Z direction) of the division line 90 from the position of the last detected division line 90 (here, the division line 90 in the search areas SA4L and SA4R). The search areas SA5L and SA5R are set at positions in the real world shifted by Dd. The setting unit 54 may set the position of the search area SA in the X direction based on the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz).

As a result, the setting unit 54 has the same position in the perpendicular direction of the lane marking 90 in the real world regardless of the increase / decrease in the vehicle speed of the vehicle 10 and the moving distance, and the lane marking 90 is periodically arranged. The search area SA will be set at a position with a high probability. After that, the setting unit 54 sets the search area SA even if the vehicle 10 moves and changes the traveling direction (for example, positions CP6 and CP7) as long as the same position in the real world for setting the search area SA is included in the captured image. The search area SA is set at the same position in the perpendicular direction of the lane marking 90 in.

Next, the coordinate system in the real world will be described.

FIG. 6 is a plan view illustrating the movement of the vehicle 10. As shown in FIG. 6, it is assumed that the vehicle 10 moves from the position CP11 to the position CP12 while turning to the right.

FIG. 7 is a plan view illustrating a coordinate system in the real world (hereinafter, rear wheel axis coordinate system) with the center of the rear wheel axis of the vehicle 10 as the origin. In the rear wheel axis coordinate system, the left-right direction of the vehicle 10 is the X direction (the left direction is the + X direction), the front-rear direction of the vehicle 10 is the Z direction (the rear is the + Z direction), and the rotation direction is the θ direction (the left rotation direction is + θ). Direction).

When the vehicle 10 moves from the position CP11 to the position CP12 as shown in FIG. 6, in the rear wheel axle coordinate system, the target point TP in the real world is as shown in FIG. 7 with respect to the vehicle 10 from the position CP11a to the position CP12a. Move relatively. In this case, the setting unit 54 sets the target point TP at the position CP12a in the X direction from the position CP11a before movement in the real world based on the wheel speed and the steering angle acquired from the wheel speed sensor 22 and the steering angle sensor 24. It is determined that the distance Zc is moved in the distance Xc and the Z direction and the angle θc is rotated in the θ direction. The setting unit 54 converts the coordinates of the real world into the coordinates of the captured image based on the movement amount of the vehicle 10 in the rear wheel axis coordinate system, the rotation angle of the vehicle 10, a preset mapping table for coordinate conversion, and the like.

FIG. 8 is a flowchart of the parking support process according to the first embodiment. The control unit 50 executes the parking support process by reading the parking support program 70 stored in the storage unit 52 based on an instruction from the occupant or the like.

As shown in FIG. 8, in the parking support process, the setting unit 54 of the control unit 50 acquires an captured image from the imaging unit 14b (S102). Next, as shown in FIG. 4, the setting unit 54 sets the search area SA in the captured image based on the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz), and sets the captured image. And the information of the search area SA is output to the detection unit 56 (S104). The detection unit 56 searches for the lane marking 90 in the search area SA set in the captured image, and determines whether or not the detection can be performed (S106). When the detection unit 56 cannot detect the division line 90 in the search area SA (S106: No), the setting unit 54 acquires a new captured image (S102) and repeats steps S104 and subsequent steps.

When the detection unit 56 detects the lane marking 90 in the search area SA (S106: Yes), the detection unit 56 outputs information such as the position and shape of the lane marking 90 to the setting unit 54. When the setting unit 54 acquires the information of the lane marking 90, it determines whether or not a plurality of lane markings 90 including the lane marking 90 are detected on the same side on the left and right (S108). If the setting unit 54 determines that the lane marking 90 detected this time is the first lane marking 90 and has not yet detected a plurality of lane markings 90 (S108: No), the setting unit 54 repeats steps S102 and subsequent steps.

On the other hand, when the setting unit 54 determines that the plurality of division lines 90 have been detected (S108: Yes), the setting unit 54 sets the parking candidate area PA between the plurality of division lines 90 as shown in FIG. 4, and the operation control unit 58. Output to (S110). The setting unit 54 calculates the detection interval Dd between the lane marking 90 and the adjacent lane marking 90 (S112).

The setting unit 54 determines whether or not a parking instruction has been obtained from the occupant (S114). For example, when the occupant touches the operation input unit 44 on the parking candidate area PA while the display unit 40 displays the image in which the parking candidate area PA is superimposed on the captured image, the setting unit 54 gives a parking instruction. Is determined to have been acquired. When the setting unit 54 determines that the parking instruction has been acquired (S114: Yes), the setting unit 54 outputs the automatic operation instruction to the operation control unit 58.

When the setting unit 54 does not acquire the parking instruction (S114: No), the setting unit 54 acquires a new captured image from the imaging unit 14b (S116). The setting unit 54 determines whether or not the search area SA can be set in the new captured image based on the detection interval Dd (S118). For example, the setting unit 54 determines whether or not a position shifted by the detection interval Dd in the perpendicular direction of the division line 90 from the position of the last set search area SA is included in the newly acquired captured image. Determines whether the search area SA can be set in the captured image. When a plurality of lane markings 90 are detected only on one of the left and right sides, the processes after step S118 may be executed only on the side where the plurality of lane markings 90 are detected.

For example, when step S116 and subsequent steps are repeated a plurality of times and the vehicle 10 moves significantly from the position where the lane marking 90 was last detected, the setting unit 54 determines that the search area SA cannot be set in the new captured image ( S118: No), step S102 and subsequent steps are executed again.

When the setting unit 54 determines that the search area SA can be set in the newly acquired captured image (S118: Yes), the setting unit 54 moves from the position of the last set search area SA by the detection interval Dd as shown in FIG. A search area SA is set in the newly acquired captured image based on the position in the real world, and is output to the detection unit 56 (S120).

The detection unit 56 searches the lane marking 90 in the search area SA and determines whether or not the detection can be performed (S122). When the division line 90 cannot be detected (S122: No), the detection unit 56 outputs to the setting unit 54 that it has not been detected. When the lane marking 90 can be detected in the search area SA (S122: Yes), the detection unit 56 outputs the information of the lane marking 90 to the setting unit 54. The setting unit 54 sets the parking candidate area PA between the newly detected lane marking 90 and the previously detected lane marking 90 (S124). When the setting unit 54 acquires from the detection unit 56 that the lane marking 90 has not been detected, or when the parking candidate area PA is set, the setting unit 54 determines whether or not a parking instruction has been acquired from the occupant (S126). When the setting unit 54 determines that the parking instruction has not been acquired (S126: No), the setting unit 54 repeats steps S116 and subsequent steps. After that, the setting unit 54 repeats steps S116 and subsequent steps as long as the parking instruction is not acquired and the search area SA can be set at a position in the captured image by the detection interval Dd.

On the other hand, when the setting unit 54 acquires the parking instruction (S126: Yes, S114: Yes), the setting unit 54 outputs the automatic operation instruction to the operation control unit 58. When the driving control unit 58 acquires the automatic driving instruction, the driving control unit 58 controls the automatic driving unit 32 to automatically drive the vehicle 10 (S130). As a result, the driving control unit 58 moves the vehicle 10 to the parking candidate area PA selected by the occupant, and ends the parking support process.

As described above, in the parking support device 36 of the first embodiment, when the detection unit 56 detects the lane marking 90, the setting unit 54 detects the lane marking SA for detecting another lane marking 90. Set based on 90. As a result, the parking support device 36 can set the search area SA for detecting the periodically arranged division lines 90 at an appropriate position in the captured image regardless of the moving direction of the vehicle 10, so that the division can be set. The detection accuracy of the line 90 can be improved.

In the parking support device 36, when the detection unit 56 detects a plurality of lane markings 90, the setting unit 54 determines another lane marking 90 based on the detection interval Dd which is the distance between the lane marking 90 and the adjacent lane marking 90. The search area SA for detecting is set. As a result, the setting unit 54 can set the search area SA for detecting the periodically arranged division lines 90 at a more appropriate position.

<Second Embodiment>
Next, a second embodiment in which the method of setting the search area SA of the first embodiment is changed will be described. FIG. 9 is a plan view illustrating the setting of the search area SA according to the second embodiment in the real world after the lane marking 90 is detected.

As shown in FIG. 9, the setting unit 54 has the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz) with the vehicle 10 at the position CP21 as the origin while the vehicle 10 is traveling at the position CP21. The search areas SA21L and SA21R are set at the positions in the real world. It is assumed that the detection unit 56 detects the lane marking 90 in the search areas SA21L and SA21R. After that, the setting unit 54 sets the position of the search area for searching another division line based on the detected position of the division line 90 and the preset interval Sd stored in the storage unit 52. To do. Specifically, when the setting unit 54 acquires a new captured image while the vehicle 10 has moved to the next position CP22, the setting unit 54 finally detects the lane marking 90 (here, the lane marking in the search areas SA21L and SA21R). The new search areas SA22L and SA22R are set at positions in the real world shifted from 90) by the preset interval Sd in the perpendicular direction (here, −Z direction) of the division line 90.

The setting interval Sd may be set based on the interval of the lane marking 90 of a general parking lot, and is, for example, a value between 1900 mm and 2400 mm. The setting unit 54 may set the position of the search area SA in the X direction based on the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz). As a result, the setting unit 54 is at the same position in the perpendicular direction of the lane marking 90 in the real world regardless of the increase / decrease in the vehicle speed of the vehicle 10 and the moving distance, and the probability that the next lane marking 90 exists is high. The search area SA will be set at the position. After that, as long as the same position in the real world is included in the captured image, the setting unit 54 searches for the same position in the perpendicular direction of the lane marking 90 in the real world even if the vehicle 10 moves (for example, position CP23). Set the area SA. When the plurality of lane markings 90 are detected, the setting unit 54 sets the parking candidate areas PAL and PAR between the plurality of lane markings 90.

FIG. 10 is a flowchart of the parking support process according to the second embodiment. For the same processing as the parking support processing of the first embodiment, the same step number is assigned to simplify the explanation.

As shown in FIG. 10, in the parking support process of the second embodiment, the setting unit 54 and the detection unit 56 execute steps S102, S104, S106, S110, S114, and S116. The setting unit 54 determines whether or not the search area SA can be set in the new captured image based on the setting interval Sd (S218). For example, the setting unit 54 determines whether or not a position shifted by the set interval Sd in the perpendicular direction of the division line 90 from the position of the last set search area SA is included in the newly acquired captured image. Determines whether the search area SA can be set in the captured image.

When the setting unit 54 determines that the search area SA cannot be set in the new captured image (S218: No), the setting unit 54 executes step S102 and subsequent steps again.

When the setting unit 54 determines that the search area SA can be set in the new captured image (S218: Yes), as shown in FIG. 9, the setting interval is set in the perpendicular direction from the last detected division line 90 to the division line 90. The search area SA is set in the new captured image based on the position in the real world shifted by Sd (S220).

The setting unit 54, the detection unit 56, and the operation control unit 58 execute step S122 and subsequent steps.

As described above, in the parking support device 36 of the second embodiment, the search area SA after detecting the lane marking 90 by the preset interval Sd set in advance based on the spacing of the lane marking 90 of a general parking lot. The position of is set. As a result, the parking support device 36 of the second embodiment can reduce the processing load as compared with the case of calculating the detection interval Dd.

<Third Embodiment>
Next, a third embodiment in which the method of setting the search area SA of the second embodiment is changed will be described. FIG. 11 is a plan view illustrating the setting of the search area SA according to the third embodiment in the real world after the lane marking 90 is detected. In the third embodiment, the setting unit 54 is set to a plurality of different positions in the real world based on the detected position of the division line 90 and the plurality of set intervals Sd1 and Sd2 preset and stored in the storage unit 52. , Set the position of the search area SA for searching other lane markings.

As shown in FIG. 11, the setting unit 54 has the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz) with the vehicle 10 at the position CP31 as the origin while the vehicle 10 is traveling at the position CP31. The search areas SA31L and SA31R are set at the positions in the real world. It is assumed that the detection unit 56 detects the lane marking 90 in the search areas SA31L and SA31R. After that, when the setting unit 54 acquires a new captured image while the vehicle 10 has moved to the next position CP32, the setting unit 54 finally detects the division line 90 (here, the division detected in the search areas SA31L and SA31R). The positions of the new search areas SA32L and SA32R are set at the positions in the real world shifted from the line 90) by the preset interval Sd1 in the perpendicular direction of the division line 90. The setting interval Sd1 may be set based on the interval of the lane marking 90 of a general parking lot, and is, for example, 2000 mm. The setting unit 54 may set the positions of the search areas SA32L and SA32R in the X direction based on the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz).

Next, when the setting unit 54 acquires a new captured image while the vehicle 10 has moved to the next position CP33, the setting unit 54 finally detects the division line 90 (here, the division detected in the search areas SA31L and SA31R). The new search areas SA33L and SA33R are set at the positions in the real world shifted from the line 90) by the preset interval Sd2 in the perpendicular direction of the division line 90. The set interval Sd2 may be set to a value different from the set interval Sd1 based on the interval of the lane marking 90 of a general parking lot, and is, for example, 2300 mm. That is, the setting unit 54 sets the search areas SA32L, SA32R, SA33L, and SA33R at a plurality of different positions in the real world based on the plurality of setting intervals Sd1 and Sd2. The setting unit 54 may set the positions of the search areas SA33L and SA33R in the X direction based on the left initial coordinates (+ Dx, Dz) and the right initial coordinates (−Dx, Dz).

After that, when the vehicle 10 moves to the position CP34 and the position CP35, the setting unit 54 alternately adopts the set interval Sd1 and the set interval Sd2, and as long as it can be set in the captured image, the search areas SA33L and SA33R in the real world. The search area SA is set at the same position as. As a result, the setting unit 54 is at the same position as the positions of the plurality of search areas SA33L and SA33R in the perpendicular direction of the lane marking 90 in the real world regardless of the increase / decrease in the vehicle speed of the vehicle 10 and the moving distance. The search area SA is set at a position where there is a high probability that the lane marking 90 is present.

Since the parking support process of the third embodiment is the same process except that a plurality of setting intervals Sd1 and Sd2 are alternately adopted in step S220 of the parking support process of the second embodiment, the description by the flowchart is omitted.

As described above, in the parking support device 36 of the third embodiment, after detecting the lane marking 90 by a plurality of preset intervals Sd1 and Sd2 based on the spacing of the lane marking 90 of a general parking lot. The position of the search area SA of is set. As a result, the parking support device 36 of the third embodiment appropriately reduces the processing load as compared with the case of calculating the detection interval Dd, and appropriately positions the search area SA even in parking lots having different intervals of the lane markings 90. Can be set.

<Fourth Embodiment>
After detecting the lane marking 90, the setting unit 54 of the fourth embodiment determines whether to set the search area SA on both sides, the left side, or the right side of the vehicle 10 by executing the left-right determination process described later. To do. Specifically, when the detected lane marking 90 is on the right side and the vehicle 10 is turning left, the setting unit 54 sets the search area SA only on the right side of the vehicle 10. Further, when the detected lane marking 90 is on the left side and the vehicle 10 is turning right, the setting unit 54 sets the search area SA only on the left side of the vehicle 10. In other cases, the setting unit 54 sets search areas SA on both sides of the vehicle 10.

The left-right determination process executed by the setting unit 54 will be described. FIG. 12 is a flowchart of the left / right determination process of the fourth embodiment. The setting unit 54 may execute the left-right determination process before step S120 or step S220, for example.

As shown in FIG. 12, when the setting unit 54 acquires the information of the lane marking 90 detected from the detection unit 56, the setting unit 54 determines whether or not the lane marking 90 is on the right side of the vehicle 10 (S302). When the setting unit 54 determines that the lane marking 90 is on the right side of the vehicle 10 (S302: Yes), the setting unit 54 acquires the steering angle detected by the steering angle sensor 24 from the in-vehicle network 38, and based on the steering angle, the vehicle It is determined whether or not 10 is turning left (S304). For example, the setting unit 54 may determine that the vehicle 10 is a left turn if the steering angle is equal to or more than a preset left turn steering angle threshold value, and may determine that the vehicle is not a left turn if it is less than the left turn steering angle threshold value. An example of the left turn steering angle threshold is + 45 °. The steering angle is positive in the counterclockwise direction and negative in the clockwise direction. When the setting unit 54 determines that the vehicle 10 has not turned left (S304: No), the setting unit 54 executes step S314 described later. On the other hand, when the setting unit 54 determines that the vehicle 10 is turning left (S304: Yes), the setting unit 54 executes the processes after step S120 or step S220 so as to set the search area SA only on the right side of the vehicle 10 (S306). To do.

On the other hand, when the setting unit 54 determines that the division line 90 detected by the detection unit 56 is not on the right side (S302: No), the setting unit 54 determines whether or not the division line 90 is on the left side (S308). When the setting unit 54 determines that the lane marking 90 is not on the left side (S308: No), the setting unit 54 executes step S314 described later. On the other hand, when the setting unit 54 determines that the lane marking 90 is on the left side (S308: Yes), the setting unit 54 acquires the steering angle detected by the steering angle sensor 24 from the in-vehicle network 38, and based on the steering angle, the vehicle 10 Determines whether or not is turning right (S310). For example, the setting unit 54 may determine that the vehicle 10 is turning right if the steering angle is less than the preset right turn steering angle threshold value, and that the vehicle 10 is not a right turn if it is larger than the right turn steering angle threshold value. An example of a right turn steering angle threshold is −45 °. When the setting unit 54 determines that the vehicle 10 has not turned right (S310: No), the setting unit 54 executes step S314 described later. On the other hand, when the setting unit 54 determines that the vehicle 10 is turning right (S310: Yes), the setting unit 54 executes the processes after step S120 or step S220 so as to set the search area SA only on the left side of the vehicle 10 (S312). To do.

In step S314, the setting unit 54 executes the processes after step S120 or step S220 so as to set the search areas SA on both sides of the vehicle 10.

In the fourth embodiment, when the setting unit 54 determines that the two conditions (the position of the lane marking 90 and the traveling direction of the vehicle 10) are satisfied, the search area SA is set only on either the left or right side. However, it is not limited to this. For example, the setting unit 54 may set the search area SA only on the right side when the vehicle 10 is turning left, and may set the search area SA only on the left side when the vehicle 10 is turning right.

As described above, in the parking support device 36 of the fourth embodiment, when the two conditions, the left and right positions of the detected lane marking 90 and the left and right traveling directions of the vehicle 10 are satisfied, only one of the left and right is satisfied. , The search area SA after detection is set. Therefore, the parking support device 36 of the fourth embodiment reduces the determination error on the side that sets the search area SA due to false detection of the lane marking 90, and whether the lane marking 90 exists on only one of the left and right sides. By improving the accuracy of whether or not and setting the search area SA on only one of the left and right sides when the condition is satisfied, the burden of processing the setting of the search area SA can be reduced.

<Fifth Embodiment>
In the fifth embodiment, the setting unit 54 sets the search area SA at a plurality of positions in the left-right direction of the vehicle 10. That is, in the above-described embodiment, the setting unit 54 sets the search area SA at the position of Dx on the left side of the vehicle 10 and sets the search area SA at the position of −Dx on the right side of the vehicle 10. In the embodiment, the search areas SA are set at a plurality of different positions on the left and right sides of the vehicle 10.

FIG. 13 is a plan view illustrating the setting of the search area SA in the real world until the setting unit 54 of the fifth embodiment detects a plurality of division lines 90. FIG. 13 assumes a case where the vehicle 10 moves from the position CP41 indicated by the alternate long and short dash line to the position CP44 indicated by the solid line via the positions CP42 and CP43 along the arrow.

The setting unit 54 of the fifth embodiment acquires the captured image at the position CP41, and based on the preset initial coordinates (+ Dx, Dz) and the initial coordinates (−Dx, Dz), the left and right sides of the vehicle 10 The search areas SA41L and SA41R are set in. Specifically, the setting unit 54 sets the search region SA41L at the left initial coordinates (+ Dx, Dz) with the position CP41 of the vehicle 10 as the origin. The setting unit 54 sets the search area SA41R at the right initial coordinates (−Dx, Dz) with the position CP41 of the vehicle 10 as the origin.

Next, the setting unit 54 acquires a new captured image at the position CP42, and sets the left initial coordinates (+ Dx, Dz), the right initial coordinates (-Dx, Dz), and the preset left / right offset value dx. Based on this, the search areas SA42L and SA42R are set on the left and right sides of the vehicle 10. Specifically, the setting unit 54 offsets the left initial coordinates (+ Dx, Dz) to the right (+ side) by the left / right offset value dx with the position CP42 of the vehicle 10 as the origin, and the left offset coordinates (+ Dx + dx, Dz). The search area SA42L is set in. The setting unit 54 sets the right initial coordinates (-Dx, Dz) to the right offset coordinates (-Dx-dx, Dz) offset to the right (-side) by the left-right offset value dx with the position CP42 of the vehicle 10 as the origin. The search area SA42R is set. That is, at the position CP42, the setting unit 54 sets the search areas SA42L and SA42R at positions in the left-right direction different from the search areas SA41L and SA41R set at the position CP41.

The setting unit 54 acquires a new captured image at the position CP43, and similarly to the position CP41, the left initial coordinates (+ Dx, Dz) and the right initial coordinates (-Dx, Dz) with the position CP43 of the vehicle 10 as the origin. The search areas SA43L and SA43R are set in. That is, the setting unit 54 sets the search area SA by repeating the processing at the position CP41 and the position CP42 until the detection unit 56 detects the division line 90.

After that, the setting unit 54 acquires a new captured image at the position CP44, and similarly to the position CP42, the left offset coordinates (+ Dx + dx, Dz) and the right offset coordinates (-Dx) with the position CP44 of the vehicle 10 as the origin. Search areas SA44L and SA44R are set in each of −dx and Dz). The detection unit 56 detects the lane marking 90 in each of the search areas SA44L and SA44R. As the subsequent processing, the same processing as that of the above-described embodiment may be executed. After detecting the lane marking 90, the setting unit 54 may apply the left-right offset value dx to the left-right position (position in the X direction) of the vehicle 10.

In the fifth embodiment, the setting unit 54 has described with reference to an example in which the search area SA is set based on one left-right offset value dx, but the present invention is not limited to this. For example, the setting unit 54 may set the search area SA based on a plurality of left-right offset values dx.

As described above, in the parking support device 36 of the fifth embodiment, since the setting unit 54 sets the search area SA at a plurality of positions in the left-right direction of the vehicle 10, the search area SA is set at a plurality of positions, regardless of the distance between the vehicle 10 and the lane marking 90. The search area SA can be set at an appropriate position to improve the detection accuracy of the lane marking 90.

<Sixth Embodiment>
In the sixth embodiment, the setting unit 54 sets the search area SA at a plurality of positions in the rear direction of the vehicle 10. That is, in the above-described embodiment, the setting unit 54 sets the search area SA at the position of Dz behind the vehicle 10, but in the sixth embodiment, the search area SA is set at a plurality of different positions behind the vehicle 10. Set.

FIG. 14 is a plan view illustrating the setting of the search area SA in the real world until the setting unit 54 of the sixth embodiment detects a plurality of division lines 90. FIG. 14 assumes a case where the vehicle 10 moves from the position CP51 indicated by the alternate long and short dash line to the position CP54 indicated by the solid line via the positions CP52 and CP53 along the arrow.

The setting unit 54 of the sixth embodiment acquires the captured image at the position CP51, and based on the preset initial coordinates (+ Dx, Dz) and the initial coordinates (−Dx, Dz), the left and right sides of the vehicle 10 The search areas SA51L and SA51R are set in. Specifically, the setting unit 54 sets the search area SA51L at the left initial coordinates (+ Dx, Dz) with the position CP51 of the vehicle 10 as the origin. The setting unit 54 sets the search region SA51R at the right initial coordinates (−Dx, Dz) with the position CP51 of the vehicle 10 as the origin.

Next, the setting unit 54 acquires a new captured image at the position CP52, and sets the left initial coordinates (+ Dx, Dz), the right initial coordinates (-Dx, Dz), and the preset front-rear offset value dz. Based on this, the search areas SA52L and SA52R are set on the left and right sides of the vehicle 10. Specifically, the setting unit 54 sets the search region SA52L in the left offset coordinates (+ Dx, Dz + dz) in which the left initial coordinates (+ Dx, Dz) are offset backward by the front-rear offset value dz with the position CP52 of the vehicle 10 as the origin. To set. The setting unit 54 sets the search area SA52R in the right offset coordinates (-Dx, Dz + dz) in which the right initial coordinates (-Dx, Dz) are offset backward by the front-rear offset value dz with the position CP52 of the vehicle 10 as the origin. .. That is, at the position CP52, the setting unit 54 sets the search areas SA52L and SA52R at positions in the front-rear direction different from the search areas SA51L and SA51R set at the position CP51.

The setting unit 54 acquires a new captured image at the position CP53, and similarly to the position CP51, the left initial coordinates (+ Dx, Dz) and the right initial coordinates (-Dx, Dz) with the position CP53 of the vehicle 10 as the origin. The search areas SA53L and SA53R are set in. That is, the setting unit 54 sets the search area SA by repeating the processing at the position CP 51 and the position CP 52 until the detection unit 56 detects the division line 90.

After that, the setting unit 54 acquires a new captured image at the position CP 54, and similarly to the position CP 52, the left offset coordinates (+ Dx, Dz + dz) and the right offset coordinates (-Dx) with the position CP 54 of the vehicle 10 as the origin. , Dz + dz), respectively, the search areas SA54L and SA54R are set. The detection unit 56 detects the lane marking 90 in each of the search areas SA54L and SA54R. As the subsequent processing, the same processing as that of the above-described embodiment may be executed.

In the sixth embodiment, the setting unit 54 has described by giving an example of setting the search area SA based on one front-back offset value dz, but the present invention is not limited to this. For example, the setting unit 54 may set the search area SA based on a plurality of front-back offset values dz.

As described above, in the parking support device 36 of the sixth embodiment, since the setting unit 54 sets the search area SA at a plurality of positions in the rear direction of the vehicle 10, the search area SA is set to an appropriate position regardless of the interval of the lane markings 90. The search area SA can be set in 1 to improve the detection accuracy of the lane marking 90.

<7th Embodiment>
In the seventh embodiment, after the setting unit 54 detects the plurality of division lines 90, the search area SA is set based on the detection interval Dd, and the search area SA is set based on the initial coordinates without using the detection interval Dd. To set.

FIG. 15 is a plan view illustrating the setting of the search area SA in the real world until the setting unit 54 of the seventh embodiment detects a plurality of division lines 90. FIG. 15 assumes a case where the vehicle 10 is moving while turning right along the arrow after the detection unit 56 detects the lane markings 90a, 90b, and 90c. In the seventh embodiment, an example in which the search area SA is set only on the left side by the left-right determination process will be described.

As shown in FIG. 15, in the setting unit 54 of the seventh embodiment, after the detection unit 56 detects the division lines 90a, 90b, 90c, the search area SA66L based on the detection interval Dd is set in each of the plurality of captured images. One of the search areas SA61L, SA62L, SA63L, SA64L, and SA65L based on the SA67L and the left initial coordinates (+ Dx, Dz) is set. That is, the setting unit 54 of the seventh embodiment sets the search areas SA61L, SA62L, SA63L, SA64L, and SA65L based on the left initial coordinates (+ Dx, Dz) even after the division line 90 is detected. As a result, after the detection unit 56 detects the division lines 90a, 90b, 90c, the division lines 90d can be detected in the search areas SA66L and SA67L, and even if the detection of the division lines 90a, 90b, 90c is erroneous detection. The search areas SA61L to SA65L can improve the possibility of detecting a new division line 90.

Here, the setting unit 54 may move the position of the search area SA back and forth by using a plurality of front-back offset values dz. Specifically, the setting unit 54 may be set in the order of the search areas SA61L, SA65L, SA62L, SA64L, and SA63L, and then in the order of the search areas SA66L and SA67L.

The functions, connection relationships, numbers, arrangements, etc. of the configurations of the above-described embodiments may be appropriately changed or deleted within the scope of the invention and the scope equivalent to the scope of the invention. Each embodiment may be combined as appropriate. The order of each step of each embodiment may be changed as appropriate.

In the above-described embodiment, an example using the captured image captured by the imaging unit 14 installed on the rear side of the vehicle 10 has been given, but the present invention is not limited to this. For example, the setting unit 54 may set the search area SA in the images captured from the imaging units 14 provided on the front side or the left and right side portions of the vehicle 10, and synthesizes the captured images acquired from the plurality of imaging units 14. The search area SA may be set in the composite image.

In the above-described embodiment, the lane marking 90 has been described with reference to an example in which the lane marking 90 is substantially orthogonal to the road of the parking lot, but the present invention is not limited thereto. For example, each of the above-described embodiments may be applied when the lane marking is inclined with respect to the parking lot road such as a parking area of a highway. In this case, the search area SA is preferably tilted with respect to the front-rear direction (X direction) of the vehicle 10. Further, when setting the lane marking 90 based on the detection interval Dd, the setting unit 54 searches for a position shifted by the detection interval Dd in the direction in which the ends of the detected plurality of lane markings 90 on the vehicle 10 side are connected. The area SA may be set.

10 ... Vehicle 14 ... Imaging unit 20 ... Parking support system 36 ... Parking support device 54 ... Setting unit 56 ... Detection unit 90 ... Section line PA ... Parking candidate area SA ... Search area Sd ... Setting interval

Claims (4)

A setting unit that sets a search area for searching for a lane marking that divides the parking area in the captured image of the surroundings of the moving vehicle.
A detection unit that searches within the search area and detects the division line,
With
The setting unit is a parking support device that sets the position of the search area for searching another section line based on the detected position of the section line and the preset setting interval . The detection unit detects a plurality of marking lines and
The parking support device according to claim 1, wherein the setting unit sets the position of the search area for searching the other lane markings based on the plurality of lane markings. The setting unit sets the position of the search area for searching the other division line at a plurality of different positions based on the detected position of the division line and the plurality of preset interval settings. The parking support device according to claim 1 . The setting unit
When the vehicle is turning left, the search area is set only on the right side of the vehicle.
The parking support device according to any one of claims 1 to 3 , wherein the search area is set only on the left side of the vehicle when the vehicle is turning right.

Images