JP7380073B2 - parking assist device - Google Patents

Description

Embodiments of the present invention relate to a parking assistance device.

2. Description of the Related Art Parking support devices that determine a parking position based on sensor detection results or captured images are conventionally known. Such a parking assist device determines a parking position by detecting a vertical marking line that is substantially parallel to the longitudinal direction of a parking area.

Japanese Patent Application Publication No. 2018-185589

Generally, the parking position in the vertical direction differs depending on the shape of the front end of the partition line. However, since the parking assist device determines the parking position based on the vertical division lines, the parking position may deviate from an appropriate position in the vertical direction. Therefore, there is a need for parking assistance that allows the vehicle to be parked at an appropriate position according to the shape of the lane markings in the parking area.

The parking assistance device according to the embodiment of the present invention includes, for example, a first detection unit that detects a marking line that is substantially parallel to the longitudinal direction of a parking area, and a position defined by the marking line detected by the first detection unit. , a setting unit that sets a region of interest that is a region for detecting edge points indicating boundaries of the partition line, and a dividing unit that divides the region of interest into a plurality of divided regions by dividing lines in an orthogonal direction perpendicular to the longitudinal direction. and a second detection unit that detects, based on the edge point in the attention area, a front end position indicating an end of the partition line that is on the entry side where a vehicle enters the parking area and is substantially parallel to the longitudinal direction. and a determining unit that determines a parking position based on the front end position. Therefore, for example, the parking assistance device can perform parking assistance to park the vehicle at an appropriate position according to the shape of the lane markings.

The parking assistance device according to the embodiment of the present invention further includes, as an example, a dividing unit that divides the attention area into a plurality of divided areas by dividing lines in an orthogonal direction perpendicular to the longitudinal direction, and the second detection unit , based on the edge points in the divided area, detect a front end position indicating an end of the partition line that is on the entry side where a vehicle enters the parking area and is substantially parallel to the longitudinal direction. Therefore, for example, the parking assistance device can detect the front end position indicating the end of the partition line from the attention area.

In the above-mentioned parking assistance device, as an example, the second detection unit identifies the front end position based on the divided area determined based on the edge point and the detection result of a marking line. Therefore, for example, the parking assist device can reduce the possibility that the parking position will shift due to noise or the like.

The above-mentioned parking assist device further includes, as an example, a determination unit that determines whether or not a lane marking line that is substantially parallel to the longitudinal direction is a double line; If it is determined that a parallel lot line is a double line, the line is within a predetermined distance from the end of the lot line, based on the lot line formed by the double line and substantially parallel to the longitudinal direction. The attention area is set in a first area, and the determination unit determines the parking position based on the front end position detected from the first area. Therefore, for example, the parking assistance device can perform parking assistance that causes the vehicle to be parked at an appropriate position according to the shape of the lane marking formed by the double line.

In the above-mentioned parking assist device, as an example, when the determination unit determines that the lane marking line substantially parallel to the longitudinal direction is not a double line, the setting unit may set the end portion of the lane marking line formed by a single line to and a third area on the side of the second area in the lateral direction of the parking area, and the determining unit is configured to When a marking line is not detected from the third area, the parking position is determined based on the front end position detected from the second area. Therefore, for example, the parking assistance device can perform parking assistance to park the vehicle at an appropriate position according to the shape of the I-shaped lane marking.

In the above-mentioned parking assistance device, as an example, when the marking line in the orthogonal direction is detected from the third area, the determining unit determines the parking position based on the front end position and the line width of the marking line. decide. Therefore, for example, the parking assistance device can perform parking assistance to park the vehicle at an appropriate position according to the shape of the T-shaped lane marking.

In the above-mentioned parking assistance device, as an example, the determination unit may be configured to detect a partition line that is substantially parallel to the longitudinal direction, and a partition line that is adjacent to the partition line that is substantially parallel to the longitudinal direction, in an orthogonal direction that is orthogonal to the longitudinal direction. If the distance is less than or equal to the threshold, it is determined that the line is a double line. Therefore, for example, the parking assist device can reduce the possibility that the marking line is not determined to be a double line.

FIG. 1 is an exemplary and schematic perspective view showing a part of the cabin of a vehicle equipped with an image processing apparatus according to an embodiment. FIG. 2 is an exemplary and schematic plan view of a vehicle equipped with the parking assist device according to the present embodiment. FIG. 3 is an exemplary and schematic diagram of the dashboard of a vehicle equipped with the parking assistance device according to the embodiment, viewed from the rear of the vehicle. FIG. 4 is an exemplary and schematic block diagram showing the functional configuration of a vehicle control system including the parking assist device according to the present embodiment. FIG. 5 is a block diagram illustratively and schematically showing the configuration of the parking assist device (parking assist unit) according to the embodiment implemented by a CPU. FIG. 6 is an explanatory diagram illustratively and schematically showing parking positions in the case of a U-shaped partition line. FIG. 7 is an explanatory diagram illustratively and schematically showing parking positions in the case of an I-shaped partition line. FIG. 8 is an explanatory diagram exemplarily and schematically showing parking positions in the case of a T-shaped partition line. FIG. 9 is an explanatory diagram illustrating and schematically showing a method for specifying the shape of the front end when the partition line is a double line. FIG. 10 is an explanatory diagram illustratively and schematically showing a method for specifying the shape of the front end when the front end of the partition line is I-shaped. FIG. 11 is an explanatory diagram illustrating and schematically showing a method for specifying the shape of the front end when the front end of the partition line is L-shaped. FIG. 12 is an explanatory diagram exemplarily and schematically showing a method for detecting the front end point when the partition line is a double line. FIG. 13 is an explanatory diagram illustratively and schematically showing an imaging area during parking. FIG. 14 is an explanatory diagram illustrating and schematically showing a method for determining whether a partition line substantially parallel to the longitudinal direction is a double line. FIG. 15 is a flowchart illustrating an example of the procedure of the parking assistance process executed by the parking assistance unit. FIG. 16 is a flowchart illustrating an example of a procedure of detection processing by the parking support unit. FIG. 17 is a flowchart illustrating an example of the procedure of determination processing by the parking support unit.

Exemplary embodiments of the invention are disclosed below. The configuration of the embodiment shown below, and the actions, results, and effects brought about by the configuration are examples. The present invention can be realized with configurations other than those disclosed in the embodiments below, and it is possible to obtain at least one of various effects based on the basic configuration and derivative effects. .

FIG. 1 is an exemplary and schematic perspective view of a state in which a part of the cabin 2a of a vehicle 1 equipped with an image processing apparatus according to an embodiment is seen through. The vehicle 1 equipped with the parking assist device according to the present embodiment may be a vehicle (internal combustion engine vehicle) that uses an internal combustion engine as a drive source, or a vehicle that uses an electric motor as a drive source (electric vehicle). The vehicle may be an automobile, a fuel cell vehicle, etc.), or a vehicle using both of these as drive sources (hybrid vehicle). Further, the vehicle 1 can be equipped with various transmission devices, and various devices (systems, parts, etc.) necessary for driving the internal combustion engine and the electric motor. Further, the system, number, layout, etc. of devices related to driving the wheels 3 in the vehicle 1 can be set in various ways.

As illustrated in FIG. 1, the vehicle body 2 of the vehicle 1 constitutes a cabin 2a in which a passenger (not shown) rides. A steering section 4, an acceleration operation section 5, a brake operation section 6, a speed change operation section 7, and the like are provided in the vehicle interior 2a, facing the seat 2b of a driver as a passenger. The steering unit 4 is, for example, a steering wheel protruding from the dashboard 24. The acceleration operation unit 5 is, for example, an accelerator pedal located under the driver's feet. The brake operation unit 6 is, for example, a brake pedal located under the driver's feet. The speed change operation unit 7 is, for example, a shift lever protruding from the center console.

Further, a display device 8 (display section) and an audio output device 9 as an audio output section are provided in the vehicle compartment 2a. The display device 8 is, for example, an LCD (Liquid Crystal Display), an OELD (Organic Electroluminescent Display), or the like. The audio output device 9 is, for example, a speaker. Further, the display device 8 is covered with a transparent operation input section 10 such as a touch panel. The passenger (user) can visually recognize the image displayed on the display screen of the display device 8 via the operation input unit 10. In addition, the occupant (driver, etc.) inputs an operation by touching, pushing, or moving the operation input unit 10 with a finger or the like at a position corresponding to the image displayed on the display screen of the display device 8. can be executed. The display device 8, the audio output device 9, the operation input unit 10, and the like are provided, for example, in a monitor device 11 located at the center of the dashboard 24 in the vehicle width direction, that is, in the left-right direction. The monitor device 11 can have an operation input unit (not shown) such as a switch, a dial, a joystick, a push button, or the like. Further, an audio output device (not shown) can be provided at a different position in the vehicle compartment 2a than the monitor device 11, and audio can be output from the audio output device 9 of the monitor device 11 and other audio output devices. be able to. Note that the monitor device 11 may be used also as a navigation system or an audio system, for example.

FIG. 3 is an exemplary and schematic diagram of the dashboard 24 of the vehicle 1 equipped with the parking assist device according to the embodiment, viewed from the rear of the vehicle. Further, a display device 12 separate from the display device 8 is provided inside the vehicle compartment 2a. As illustrated in FIG. 3, the display device 12 is provided, for example, in the instrument panel section 25 of the dashboard 24, approximately in the center of the instrument panel section 25, between the speed display section 25a and the rotation speed display section 25b. It is located in The size of the screen 12a of the display device 12 is smaller than the size of the screen 8a of the display device 8 (see FIG. 3). The display device 12 can display images mainly showing information related to parking assistance for the vehicle 1 (text information, information displayed by indicators, etc.). The amount of information displayed on the display device 12 may be less than the amount of information displayed on the display device 8. The display device 12 is, for example, an LCD, an OELD, or the like. Note that the information displayed on the display device 12 may be displayed on the display device 8.

FIG. 2 is an exemplary and schematic plan view of the vehicle 1 equipped with the parking assist device according to the present embodiment. As shown in FIGS. 1 and 2, the vehicle 1 is a four-wheel vehicle or the like, and has two left and right front wheels 3F and two left and right rear wheels 3R. All or some of the four wheels 3 are steerable.

The vehicle body 2 is provided with, for example, four imaging units 15a to 15d as the plurality of imaging units 15. The imaging unit 15 is, for example, a digital camera incorporating an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor). The imaging unit 15 can output video data at a predetermined frame rate. The imaging units 15 each have a wide-angle lens or a fisheye lens, and can image a range of, for example, 140° to 220° in the horizontal direction. Further, the optical axis of the imaging unit 15 is set diagonally downward. Therefore, the imaging unit 15 sequentially photographs the external environment around the vehicle body 2, including the road surface on which the vehicle 1 can move and the area where the vehicle 1 can be parked, and outputs the images as captured image data.

The imaging unit 15a is located, for example, at the rear end 2e of the vehicle body 2, provided on a wall below the trunk door 2h, and images the situation in the rear area of the vehicle 1. The imaging unit 15b is located, for example, at the right end 2f of the vehicle body 2, is provided on the right side door mirror 2g, and images the situation in an area including the right front, right side, and right rear of the vehicle 1. The imaging unit 15c is located, for example, at the front side of the vehicle body 2, that is, at the front end 2c in the vehicle longitudinal direction, is provided on a front bumper, etc., and images the situation in the front area of the vehicle 1. The imaging unit 15d is located, for example, on the left side of the vehicle body 2, that is, on the left end 2d in the vehicle width direction, and is provided on the door mirror 2g as a left side protrusion, and is provided on the left front, left side, and left rear of the vehicle 1. image the situation in the area including the area. The ECU 14 (see FIG. 4) constituting the image processing device executes arithmetic processing and image processing based on the captured image data obtained by the plurality of imaging units 15, and generates an image with a wider viewing angle, and It is possible to generate a virtual overhead image of 1 viewed from above (directly above or diagonally above).

The vehicle 1 also includes a plurality of distance measuring units 16 and 17 that can measure distances to objects existing outside the vehicle 1. The distance measuring unit 16 is, for example, a millimeter wave radar or the like, and is capable of measuring the distance to an object existing in the traveling direction of the vehicle 1 (the direction in which the vehicle 1 is facing). In this embodiment, the vehicle 1 has a plurality of distance measuring units 16a to 16d. The distance measuring unit 16a is provided, for example, at the left end of the rear bumper of the vehicle 1, and is capable of measuring the distance to an object present at the left rear of the vehicle 1. Further, the distance measuring section 16b is provided at the right end of the rear bumper of the vehicle 1, and is capable of measuring the distance to an object present on the right rear side of the vehicle 1. The distance measuring section 16c is provided at the right end of the front bumper of the vehicle 1, and is capable of measuring the distance to an object present in the right front of the vehicle 1. Further, the distance measuring section 16d is provided at the left end of the front bumper of the vehicle 1, and is capable of measuring the distance to an object present on the left front of the vehicle 1. The distance measuring section 16 can be used to detect objects over a relatively long distance.

The vehicle 1 also includes a distance measuring section 17 that can measure the distance to an external object that is relatively close to the vehicle 1. The distance measuring unit 17 is, for example, a sonar that emits ultrasonic waves and captures the reflected waves. In this embodiment, the vehicle 1 has a plurality of distance measuring sections 17a to 17h. The distance measuring units 17a to 17d are provided on the rear bumper of the vehicle 1, and are capable of measuring distances to objects present behind the vehicle 1. The distance measuring units 17e to 17h are provided on the front bumper of the vehicle 1 and can measure the distance to an object present in front of the vehicle 1.

In this embodiment, for example, when parking the vehicle 1, the distance measuring unit 16 detects obstacles that are lined up with the vehicle 1 (for example, adjacent vehicles, walls, etc.) or obstacles that exist on the back side of the parking space (for example, , curbs, steps, walls, fences, etc.) and can measure the distance to the obstacle. Further, when an obstacle (object) approaches the vehicle 1 by more than a predetermined distance (for example, 0.3 m), the distance measuring unit 17 detects the approaching obstacle (object) and detects the approaching obstacle (object). Distance to obstacles can be measured. In particular, the distance measuring units 17a and 17d disposed on both rear sides of the vehicle 1 measure the distance between the rear corner of the vehicle 1 and an obstacle (for example, an adjacent vehicle) when the vehicle 1 enters a parking space while backing up. After entering the vehicle, it also functions as a sensor (clearance sonar) that measures the distance between the rear corner and obstacles such as walls. The ECU 14 can measure the presence or absence of an object such as an obstacle located around the vehicle 1 and the distance to the object based on the detection results of the distance measuring units 16 and 17. That is, the distance measuring units 16 and 17 detect objects (stationary objects and moving objects) existing around the vehicle 1. Stationary objects include parked vehicles, walls, curbs, street trees, etc., and moving objects include running vehicles, bicycles, pedestrians, animals, etc.

FIG. 4 is an exemplary and schematic block diagram showing the functional configuration of the control system of the vehicle 1 including the parking assist device according to the present embodiment. As illustrated in FIG. 4, the control system includes an ECU 14, a monitor device 11, distance measuring units 16, 17, etc., as well as a steering system 13, a brake system 18, a steering angle sensor 19, an accelerator sensor 20, a shift sensor 21, Wheel speed sensors 22, drive system 23, etc. are electrically connected via an in-vehicle network 26 as a telecommunications line. The in-vehicle network 26 is configured as, for example, a CAN (Controller Area Network). The ECU 14 can control the steering system 13, the brake system 18, the drive system 23, etc. by sending control signals through the in-vehicle network 26.

In addition, the ECU 14 receives the detection results of the torque sensor 13b, brake sensor 18b, steering angle sensor 19, distance measuring sections 16, 17, accelerator sensor 20, shift sensor 21, wheel speed sensor 22, etc. via the in-vehicle network 26, It is possible to receive operation signals and the like from the operation input section 10.

The steering system 13 is an electric power steering system, an SBW (Steer By Wire) system, or the like. Steering system 13 includes an actuator 13a and a torque sensor 13b. The steering system 13 is electrically controlled by the ECU 14 or the like, and operates the actuator 13a to add torque to the steering unit (steering wheel, etc.) to supplement the steering force, thereby turning the wheels 3. steer. The torque sensor 13b detects the torque that the driver applies to the steering section 4, and transmits the detection result to the ECU 14.

The brake system 18 includes an ABS (Anti-lock Brake System) that controls the locking of the brakes of the vehicle 1, an electronic stability control (ESC) that suppresses skidding of the vehicle 1 during cornering, and an electronic stability control (ESC) that increases the braking force. It includes an electric brake system that assists the brakes and BBW (Brake By Wire). Brake system 18 includes an actuator 18a and a brake sensor 18b. The brake system 18 is electrically controlled by the ECU 14 and the like, and applies braking force to the wheels 3 via the actuator 18a. The brake system 18 detects signs of brake locking, idling of the wheels 3, skidding, etc. from the difference in rotation between the left and right wheels 3, and performs control to suppress locking of the brakes, idling of the wheels 3, and skidding. Execute. The brake sensor 18b is a displacement sensor that detects the position of a brake pedal as a movable part of the brake operation unit 6, and transmits the detection result of the position of the brake pedal to the ECU 14.

The steering angle sensor 19 is a sensor that detects the amount of steering of the steering unit 4 such as a steering wheel. The steering angle sensor 19 is composed of a Hall element or the like, detects the rotation angle of the rotating portion of the steering section 4 as a steering amount, and transmits the detection result to the ECU 14. The accelerator sensor 20 is a displacement sensor that detects the position of an accelerator pedal as a movable part of the acceleration operation section 5, and transmits the detection result to the ECU 14.

The shift sensor 21 is a sensor that detects the position of a movable part (bar, arm, button, etc.) of the speed change operation section 7, and transmits the detection result to the ECU 14. The wheel speed sensor 22 is a sensor that includes a Hall element and the like and detects the amount of rotation of the wheel 3 and the number of rotations of the wheel 3 per unit time, and transmits the detection result to the ECU 14.

The drive system 23 is an internal combustion engine system or a motor system as a drive source. The drive system 23 controls the fuel injection amount and intake amount of the engine and the output value of the motor according to the amount of operation required by the driver (user) (for example, the amount of depression of the accelerator pedal) detected by the accelerator sensor 20. . Moreover, the output values of the engine and motor can be controlled in cooperation with the control of the steering system 13 and the brake system 18, depending on the running state of the vehicle 1, regardless of the user's operation. For example, normal driving assistance including parking assistance can be performed.

Note that the configuration, arrangement, electrical connection form, etc. of the various sensors and actuators described above are merely examples, and can be variously set (changed).

The ECU 14 is composed of a computer or the like, and controls overall control of the vehicle 1 through cooperation of hardware and software. Specifically, the ECU 14 includes a CPU (Central Processing Unit) 14a, a ROM (Read Only Memory) 14b, a RAM (Random Access Memory) 14c, a display control section 14d, an audio control section 14e, and an SSD (Solid State Drive) 14f. Equipped with The CPU 14a, ROM 14b, and RAM 14c may be provided within the same circuit board.

The CPU 14a can read a program installed and stored in a non-volatile storage device such as a ROM 14b, and perform arithmetic processing according to the program. The CPU 14a can execute a process of recognizing the surrounding situation of the vehicle 1 and the parking space (parking position), for example, based on the surrounding information acquired by the imaging section 15, the distance measuring sections 16, 17, etc. Further, the CPU 14a performs a process of extracting changes in the surrounding situation based on the surrounding information around the vehicle 1 acquired at different timings, a process of estimating the current position of the vehicle 1 based on the surrounding information, and a process of extracting changes in the surrounding information. It is possible to perform a process of storing the reflected information, a process using the stored information, a process of guiding the vehicle 1 to the parking position, and the like. In addition, when displaying the image captured by the imaging unit 15 on the display device 8, the CPU 14a performs arithmetic processing and image processing on the captured image (curved image) of the wide-angle image obtained by the imaging unit 15 to correct distortion. or generate an overhead image (surrounding image) in which a vehicle image (self-vehicle icon) representing the vehicle 1 is displayed at the center position, for example, based on the captured image captured by the imaging unit 15. I can do it. Furthermore, when generating the bird's-eye view image, the CPU 14a can change the position of the virtual viewpoint to generate a bird's-eye view image that looks at the vehicle image from directly above or an overhead image that looks at the vehicle image from an oblique direction. .

The ROM 14b stores various programs and parameters necessary for executing the programs. The RAM 14c temporarily stores various data used in calculations by the CPU 14a. Among the arithmetic processing performed by the ECU 14, the display control unit 14d mainly performs image processing on captured image data acquired from the imaging unit 15 and output to the CPU 14a, and display for displaying image data acquired from the CPU 14a on the display devices 8 and 12. Executes conversion to image data for use. Among the calculation processing performed by the ECU 14, the audio control unit 14e mainly executes audio processing that is acquired from the CPU 14a and output to the audio output device 9. The SSD 14f is a rewritable nonvolatile storage unit that continues to store data acquired from the CPU 14a even when the ECU 14 is powered off. Note that the CPU 14a, ROM 14b, RAM 14c, etc. can be integrated in the same package. Further, the ECU 14 may have a configuration in which another logical operation processor such as a DSP (Digital Signal Processor), a logical circuit, or the like is used instead of the CPU 14a. Further, an HDD (Hard Disk Drive) may be provided in place of the SSD 14f, or the SSD 14f and the HDD may be provided separately from the ECU 14.

FIG. 5 is a block diagram illustratively and schematically showing a configuration when the parking assist device (parking assist unit 140) according to the embodiment is implemented by the CPU 14a. By executing the parking assistance program read from the ROM 14b, the CPU 14a executes the data acquisition section 141, vertical line detection section 142, attention area setting section 143, edge detection section 144, and attention area division section 145, as shown in FIG. , a front end point detection section 146, a double line determination section 147, and a parking position determination section 148. Note that the data acquisition unit 141, vertical line detection unit 142, attention area setting unit 143, edge detection unit 144, attention area division unit 145, front end point detection unit 146, double line determination unit 147, parking position determination unit 148, etc. may be partially or entirely configured by hardware such as a circuit. Although not shown in FIG. 5, the CPU 14a can also implement various modules necessary for the vehicle 1 to travel. Although FIG. 5 mainly shows the CPU 14a that executes the parking assistance process, it may also include a CPU for implementing various modules necessary for the vehicle 1 to run, or it may include an ECU separate from the ECU 14. It's okay.

The data acquisition unit 141 acquires captured image data captured by the imaging unit 15.

The vertical line detection unit 142 detects, from the captured image data acquired by the data acquisition unit 141, a division line that is substantially parallel to the longitudinal direction of the parking area, among the division lines indicating the parking position. That is, the vertical line detection unit 142 detects a vertical marking line that is substantially parallel to the longitudinal direction of the parking area. More specifically, the vertical line detection unit 142 scans in the horizontal direction perpendicular to the vertical direction and applies a filter to the captured image data to detect differences in brightness between pixels. As a result, the vertical line detection unit 142 detects a black-and-white edge point P11 that changes from black to white. Similarly, the vertical line detection unit 142 detects a black-and-white edge point P12 that changes from white to black from the captured image data. Note that if the black-and-white edge point P11 and the black-and-white edge point P12 are not to be distinguished, they will be referred to as edge point P1.

When the edge point P1 is detected, the vertical line detection unit 142 removes noise based on processing such as RANSAC (Random Sample Consensus) and CONSAC (Connected Sample Consensus). Here, in RANSAC and CONSAC, two edge points P1 are randomly selected. Then, edge points P1 in a band-shaped area within a certain range from the line connecting the two edge points P1 are counted. By repeatedly executing this process, a band-shaped area having the maximum number of edge points P1 is identified. Here, when the edge point P1 is detected from the captured image data, the edge point P1 unrelated to the partition line is also detected. Therefore, by executing processing such as RANSAC or CONSAC, noise is removed and the rough position of the vertical partition line is specified. Further, the vertical line detection unit 142 uses the least squares method to draw a straight line that has the shortest distance to each edge point P1 in the band-shaped region where the number of edge points P1 is the maximum. The vertical line detection unit 142 detects a vertical partition line by performing this process on both the black-and-white edge point P11 and the black-and-white edge point P12.

Further, the vertical line detection unit 142 detects a temporary front end point by detecting a marking line that is substantially parallel to the longitudinal direction of the parking area. Here, the provisional front end point is a marking line that is substantially parallel to the longitudinal direction of the entry side where the vehicle 1 enters the parking area among the marking lines that are substantially parallel to the longitudinal direction of the parking area detected by the vertical line detection unit 142. It's the edge. Then, the temporary front end point is provisionally set before executing the processing described later.

By the way, the parking position of the vehicle 1 generally differs depending on the shape of the front end of the lane marking. The front end portion is an end portion of a partition line that is substantially parallel to the longitudinal direction of the parking area, and is substantially parallel to the longitudinal direction of the entry side where the vehicle 1 enters the parking area. The vertical line detection unit 142 detects a vertical marking line among marking lines indicating a parking position. That is, the vertical line detection unit 142 cannot detect anything other than the vertical partition line. Therefore, depending on the shape of the front end of the lane marking, it may not be possible to identify an appropriate parking position.

Here, FIG. 6 is an explanatory diagram exemplarily and schematically showing parking positions in the case of a U-shaped partition line. FIG. 7 is an explanatory diagram illustratively and schematically showing parking positions in the case of an I-shaped partition line. FIG. 8 is an explanatory diagram exemplarily and schematically showing parking positions in the case of a T-shaped partition line.

As shown in FIG. 6, the U-shaped partition line L1, which has a U-shaped front end, has a straight line portion L11 that is a double straight line, and a U-shaped portion L12 that has a U-shape. There is. Furthermore, in the case of the U-shaped marking line L1, it is required that the vehicle 1 be parked so that the leading end of the vehicle 1 is located at the leading end of the U-shaped portion L12 rather than the leading end of the straight portion L11. However, since the vertical line detection unit 142 detects the vertical partition line among the partition lines, it cannot detect the U-shaped portion L12. Therefore, conventionally, the vehicle 1 has been parked so that the front end of the vehicle 1 is located at the front end of the straight line portion L11.

Further, as shown in FIG. 7, in the case of an I-shaped marking line L2 whose front end is an I-shaped marking line, the vehicle 1 must be parked so that the leading edge of the vehicle 1 is at the leading edge of the I-shaped marking line L2. is required. Further, as shown in FIG. 7, a T-shaped marking line L3 whose front end is a T-shaped marking line has a vertical line portion L31 that is substantially parallel to the longitudinal direction of the parking area and a vertical line portion L31 that is perpendicular to the longitudinal direction of the parking area. It has a horizontal line portion L32 parallel to . In the case of the T-shaped marking line L3, the vehicle 1 is required to be parked so that the front end of the vehicle 1 is located inside the horizontal line portion L32. That is, it is required that the vehicle 1 be parked so that its front end is further back than in the case of the I-shaped lane marking L2.

Therefore, the parking support unit 140 sets a region of interest R1 (see FIGS. 9, 10, and 11) at the position determined by the detection result of the vertical line detection unit 142 in order to specify the shape of the front end of the lane marking. do. The parking support unit 140 also identifies the shape of the front end by scanning the region of interest R1. Then, the parking support unit 140 determines the parking position according to the shape of the front end of the specified lane marking.

The process of specifying the shape of the front end by the attention area setting unit 143, the edge detection unit 144, and the attention area dividing unit 145 will be described with reference to FIGS. 9 and 10. FIG. 9 is an explanatory diagram illustrating and schematically showing a method for specifying the shape of the front end when the partition line is a double line. FIG. 10 is an explanatory diagram illustratively and schematically showing a method for specifying the shape of the front end when the front end of the partition line is I-shaped. FIG. 11 is an explanatory diagram illustrating and schematically showing a method for specifying the shape of the front end when the front end of the partition line is L-shaped.

The attention area setting unit 143 sets an attention area R1, which is an area where an edge point P1 indicating the boundary of the lane marking is detected, at a position defined by the lane marking substantially parallel to the longitudinal direction of the parking area detected by the vertical line detection unit 142. Set. First, a case will be described in which a double line determination unit 147 (described later) determines that a partition line substantially parallel to the longitudinal direction is a double line. As shown in FIG. 9, when the double line determining unit 147 determines that the partition line that is substantially parallel to the longitudinal direction is a double line, the attention area setting unit 143 determines that the longitudinal direction formed by the double line is a double line. A region of interest R1 is set in a first area within a predetermined distance from the end of the partition line, based on the partition line that is substantially parallel to the partition line. That is, the attention area setting unit 143 sets the attention area R1 for the double line. Further, the first area may be an area adjacent to the end of the partition line that is substantially parallel to the longitudinal direction detected by the vertical line detection unit 142, or may be an area an arbitrary distance away from the end. Alternatively, it may be a region including an end.

Further, when the double line determining unit 147 determines that the partition line substantially parallel to the longitudinal direction is not a double line, the attention area setting unit 143 sets a second area including the end of the partition line formed by a single line. A region of interest R1 is set in the region and a third region on the side of the second region. More specifically, as shown in FIG. 10, when the double line determining unit 147 determines that the dividing line substantially parallel to the longitudinal direction is not a double line, the attention area setting unit 143 determines whether the area of interest is formed by a single line or not. A second region including the end of the partition line is set as the region of interest R1. That is, the attention area setting unit 143 sets the attention area R1 for the I-shape. Similar to the first area, the second area may be an area adjacent to the end of the partition line that is substantially parallel to the longitudinal direction detected by the vertical line detection unit 142, or may be an area located an arbitrary distance away from the end. It may be a region or a region including an end.

Further, after detecting the front end point P2 (see FIG. 12) from the second area, as shown in FIG. 3 areas are set as the attention area R1. That is, the attention area setting unit 143 sets the attention area R1 for the T-shape. Thereby, the parking support unit 140 detects the marking line in the lateral direction of the parking area. Note that the third region may be a region adjacent to the second region, a region separated by an arbitrary distance from the second region, or a region including the second region. Further, the third region may be both regions on the sides of the second region, or may be one of the regions.

The edge detection unit 144 detects an edge point P1 indicating the boundary of the partition line from the region of interest R1. More specifically, the edge detection unit 144 detects the edge point P1 using a vertical filter that scans in the longitudinal direction of the parking area and a horizontal filter that scans in an orthogonal direction orthogonal to the longitudinal direction. Thereby, the edge detection unit 144 detects an edge point P1 whose color changes in two directions: the longitudinal direction and the orthogonal direction perpendicular to the longitudinal direction. In FIGS. 9 to 11, an open circle indicates a black-and-white edge point P12 that changes from white to black. Further, a black circle indicates a black-and-white edge point P11 where the color changes from black to white.

The attention area dividing unit 145 divides the attention area R1 into a plurality of divided areas using dividing lines in an orthogonal direction orthogonal to the longitudinal direction of the parking area. That is, as shown in FIGS. 9 to 11, the region of interest R1 is divided into divided regions.

The front end point detection unit 146 detects a front end position indicating the end of a partition line that is on the entry side where the vehicle 1 enters the parking area and is substantially parallel to the longitudinal direction, based on the edge point P1 of the region of interest R1. In other words, the front end point detection unit 146 detects a marking line on the entry side where the vehicle 1 enters the parking area and which is substantially parallel to the longitudinal direction, based on the edge point P1 in the divided area divided by the attention area dividing unit 145. Detect the front end position indicating the end. For example, the front end point detection unit 146 detects the front end point P2 as a point indicating the front end position. The front end point P2 is a point indicating the longitudinal end of the partition line in the region of interest R1. The coordinates on the attention area R1 indicated by the front end point P2 are applied to the coordinates on the parking area. Then, a parking position determination unit 148, which will be described later, determines the parking position based on the front end point P2. Note that the front end position is not limited to the front end point P2, and may be a line indicating the front end position, a surface indicating the front end position, or another shape may indicate the front end position.

More specifically, the front end point detection unit 146 detects an edge peak, which is a line indicating the longitudinal end of the partition line substantially parallel to the longitudinal direction, based on the number of edge points P1 in the divided region. Specifically, as shown in FIG. 9, the front end point detection unit 146 detects the divided region having the most black and white edge points P11. Thereby, the front end point detection unit 146 detects the inner end of the U-shape. Further, the front end point detection unit 146 detects the divided area having the most black and white edge points P12. Thereby, the front end point detection unit 146 detects the outer end of the U-shape. The front end point detection unit 146 determines that a U-shaped partition line has been detected when the distance from the inner edge to the outer edge of the U-shape is approximately the same as the line width of the partition line. That is, the front end point detection unit 146 detects an edge peak from the divided region where the black and white edge points P12 are the largest. In this manner, the front end point detection unit 146 can reduce the possibility of erroneously detecting an edge peak due to noise by determining whether the line is a U-shaped partition line. Note that the front end point detection unit 146 is not limited to the number of edge points P1, and may detect edge peaks based on a ratio, edge peaks based on density, or edge peaks based on other matters. You may.

The front end point detection unit 146 specifies the front end position based on the divided area defined based on the edge point P1 and the detection result of the partition line. Specifically, the position where the divided area determined based on the edge point P1 intersects with the center line located at the center of the outer side of the partition line that is substantially parallel to the longitudinal direction is specified as the front end point P2. That is, the front end point detection unit 146 detects the point where the edge peak and the center line are perpendicular to each other as the front end point P2. Here, FIG. 12 is an explanatory diagram illustratively and schematically showing a method of detecting the front end point P2 when the partition line is a double line. As shown in FIG. 12, in the case of a double marking line, the front end point detection unit 146 determines the point where the center line of the outer side of the pair of opposing marking lines and the edge peak are perpendicular to each other as the front end point P2. To detect.

Further, in the case of a single lane marking line, the front end point detection unit 146 detects an edge peak, which is a line indicating an end of the marking line substantially parallel to the longitudinal direction, based on the number of edge points P1 in the divided area. Specifically, as shown in FIG. 10, the front end point detection unit 146 detects the divided area in which the black and white edge points P12 are the largest. Thereby, the front end point detection unit 146 detects an I-shaped edge peak. Then, the front end point detection unit 146 detects the point where the center line of the outer side of the opposing partition line and the edge peak intersect at right angles as the front end point P2.

Further, the front end point detection unit 146 detects a T-shaped partition line. That is, the front end point detection unit 146 detects a horizontal marking line that is substantially parallel to the lateral direction of the parking area. Specifically, as shown in FIG. 11, the front end point detection unit 146 detects the divided area in which the number of black and white edge points P11 is the largest. Thereby, the front end point detection unit 146 detects the inner end of the partition line in the transverse direction. Further, the front end point detection unit 146 detects the divided area having the most black and white edge points P12. Thereby, the front end point detection unit 146 detects the outer end of the partition line in the transverse direction. Then, the front end point detection unit 146 determines that a transverse partition line has been detected when the distance from the inner edge to the outer edge of the transverse partition line is approximately the same as the width of the partition line. judge. In this manner, the front end point detection unit 146 can reduce the possibility of erroneously detecting an edge peak due to noise by determining whether it is a partition line in the transverse direction. Note that the front end point detection unit 146 is not limited to the number of edge points P1, and may detect edge peaks based on a ratio, edge peaks based on density, or edge peaks based on other matters. You may. Then, the front end point detection unit 146 changes the position of the front end point P2 to the inside of the parking area by an amount corresponding to the line width of the partition line in the transverse direction.

The double line determination unit 147 determines whether a partition line substantially parallel to the longitudinal direction of the parking area is a double line. Here, FIG. 13 is an explanatory diagram illustratively and schematically showing the imaging area A1 at the time of parking. As shown in FIG. 13, even when the vehicle 1 enters the parking area, the double line determination unit 147 determines whether or not the marking line is a double line. When the vehicle 1 enters the parking area by stepping on the marking line, the end of the marking line is outside the field of view of the imaging unit 15. That is, the end of the partition line is outside the imaging area A1, which is the area imaged by the imaging unit 15. Therefore, the vertical line detection unit 142 detects the temporary front end of the partition line from within the imaging area A1, rather than the original end of the partition line. Therefore, as shown in FIG. 13, the line connecting the two temporary front end points is directed not in the lateral direction of the parking area but in the diagonal direction. Therefore, if the double line determination unit 147 determines whether or not a partition line substantially parallel to the longitudinal direction is a double line based on the distance from one temporary front end point to the other temporary front end point, the double line determination unit 147 makes an erroneous determination. There is a possibility that you may end up doing this.

Here, FIG. 14 is an explanatory diagram illustrating and schematically showing a method for determining whether a partition line substantially parallel to the longitudinal direction is a double line. As shown in FIG. 14, the double line determination unit 147 decomposes the distance into a vertical distance and a horizontal distance. Then, the double line determination unit 147 determines whether or not it is a double line based on whether the lateral distance is less than or equal to a threshold value. In other words, the double line determination unit 147 determines whether the double line is a double line when the distance in the orthogonal direction between a lot line that is substantially parallel to the longitudinal direction and an adjacent lot line that is substantially parallel to the longitudinal direction is less than or equal to the threshold value. It is determined that

The parking position determination unit 148 determines the parking position based on the front end point P2. More specifically, the parking position determination unit 148 determines the position where the front end point of the vehicle 1 matches the position indicated by the front end point P2 detected by the front end point detection unit 146 as the parking position. Furthermore, when the marking line is U-shaped, the parking position determination unit 148 determines the parking position based on the front end point P2 detected from the first region in which the attention region R1 is set. In addition, when the marking line is I-shaped, if the parking position determination unit 148 does not detect a peak edge from the third region in which the attention region R1 is set, the parking position determination unit 148 determines the parking position based on the front end point P2 detected from the second region. to determine the parking position. That is, the parking position determining unit 148 determines the parking position based on the front end point P2 detected from the second area when the partition line in the orthogonal direction perpendicular to the longitudinal direction is not detected from the third area. In addition, when the marking line is T-shaped, when the parking position determination unit 148 detects a marking line in an orthogonal direction perpendicular to the longitudinal direction from the third area, the parking position determination unit 148 determines the front end point P2 and the line width of the marking line. Based on this, the parking position is determined. That is, the parking position determination unit 148 determines, as the parking position, a position where the position of the front end point P2 detected from the second area is changed to the inside of the parking area by the line width of the partition line in the lateral direction.

Next, an example of the parking support process executed by the parking support unit 140 will be described. FIG. 15 is a flowchart illustrating an example of the procedure of the parking assistance process executed by the parking assistance unit 140.

The data acquisition unit 141 acquires input data such as information about the vehicle 1 and captured image data (step S1).

The parking support unit 140 recognizes the lane markings (step S2). That is, the parking support unit 140 recognizes the lane markings through the processing shown in FIGS. 16 and 17.

The parking support unit 140 manages the recognized lane markings (step S3). For example, the parking support unit 140 updates the position of the detected lane marking.

The parking support unit 140 calculates the parking position using the front end point P2 etc. detected by lane marking recognition etc. (step S4).

The parking support unit 140 outputs information indicating the parking position to the ECU 14 such as a display system and a control system (step S5).

With the above, the parking support unit 140 ends the parking support process.

Next, an example of a detection process in which the parking support unit 140 detects the front end point P2 will be described. FIG. 16 is a flowchart illustrating an example of the procedure of the detection process by the parking support unit 140.

The vertical line detection unit 142 detects a straight line from the captured image data acquired by the data acquisition unit 141 (step S11). That is, the vertical line detection unit 142 detects a vertical marking line that is substantially parallel to the longitudinal direction of the parking area.

The vertical line detection unit 142 detects a straight line as a pair of the detected straight line (step S12). That is, the vertical line detection unit 142 detects a partition line that faces the detected partition line.

The double line determination unit 147 determines whether a double line is formed by the two detected partition lines (step S13).

If the partition line is a double line (step S14; Yes), the attention area setting unit 143 sets a double line attention area R1 at the end of the partition line formed by the double line (step S15). ). The front end point detection unit 146 detects the front end point P2 of the U-shaped partition line from the region in which the region of interest R1 is set (step S16). The parking support unit 140 then ends the detection process.

If the marking line is a single line (step S14; No), the attention area setting unit 143 sets the attention area R1 for the letter I at the end of the marking line formed by the single line (step S17). The front end point detection unit 146 detects the front end point P2 of the I-shaped partition line from the region where the attention region R1 is set (step S18).

The attention area setting unit 143 sets a T-shaped attention area R1 in a region to the side of the area where the attention area R1 was set in step S17 (step S19).

The front end point detection unit 146 determines whether or not an edge peak substantially parallel to the lateral direction of the parking area is detected from the area where the T-shaped attention area R1 is set (step S20). That is, the front end point detection unit 146 determines whether an edge peak due to the row of black and white edge points P11 and an edge peak due to the row of black and white edge points P12 are detected. If an edge peak is not detected (step S20; No), the detection process by the parking support unit 140 ends.

When an edge peak is detected (step S20; Yes), the front end point detection unit 146 determines whether the width between the edge peaks is substantially the same as the line width of the partition line (step S21).

The front end point detection unit 146 detects that when the width between the edge peaks is approximately the same as the line width of the partition line,
The position of the front end point P2 is changed to the inside of the parking area by the line width of the lane marking line in the transverse direction (step S22).

With the above, the parking support unit 140 ends the detection process. The parking support unit 140 can park the vehicle 1 at an appropriate position according to the shape of the lane marking by setting the position indicated by the front end point P2 detected by the detection process as the parking position.

Next, an example of a determination process executed by the parking support unit 140 to determine whether or not a lane marking is a double line will be described. FIG. 17 is a flowchart illustrating an example of the procedure of the determination process by the parking support unit 140.

The double line determination unit 147 calculates the lateral distance from the temporary front end point detected from the captured image data to the temporary front end point (step S31). That is, as shown in FIG. 14, the distance in the transverse direction from the provisional front end point of the partition line substantially parallel to the longitudinal direction to the partition line substantially parallel to the longitudinal direction where the other provisional front end point was detected is calculated.

The double line determination unit 147 calculates the vertical distance from the temporary front end point detected from the captured image data to the temporary front end point (step S32). That is, as shown in FIG. 14, the distance in the longitudinal direction from the temporary front end point to the other temporary front end point is calculated.

The double line determination unit 147 determines whether or not the division line to be determined has been registered as a double line (step S33). Here, the double line determination unit 147 also determines whether the lane marking is a double line when the vehicle passes in front of the lane marking. Further, the double line determination unit 147 determines whether the marking line is a double line even when the vehicle is parked. The accuracy of determining the positional relationship differs depending on the distance between the vehicle 1 and the marking line. Therefore, the double line determination unit 147 improves accuracy by performing determination intermittently.

If it has been registered that the lot line to be determined is a double line (step S33; Yes), the double line determination unit 147 determines whether the lateral distance from the temporary front end point to the temporary front end point is less than a threshold value. It is determined whether or not (step S34).

If the lateral distance from the temporary front end point to the temporary front end point is less than the threshold (step S34; Yes), the double line determination unit 147 determines that the marking line is a double line (step S35). Then, the parking support unit 140 ends the determination process.

If the lateral distance from the temporary front end point to the temporary front end point is not less than the threshold (step S34; No), the double line determination unit 147 determines that the marking line is a single line (step S36). Then, the parking support unit 140 ends the determination process.

If it has not been registered that the lot line to be determined is a double line (step S33; No), the double line determination unit 147 determines that the horizontal distance from the temporary front end point to the temporary front end point is less than the threshold and that the temporary It is determined whether the vertical distance from the front end point to the temporary front end point is less than a threshold (step S37). Note that the vertical distance threshold is a small value compared to the horizontal distance threshold. Here, the double line determination unit 147 also determines whether the lane marking is a double line when the vehicle passes in front of the lane marking. In other words, the case where the double line is not registered is assumed to be the case where the lane marking is detected when the vehicle passes in front of the lane marking. Therefore, as shown in FIG. 13, the vehicle 1 is not stepping on the marking line. Therefore, it is required that there be almost no difference in vertical distance.

If the conditions that the horizontal distance is less than the threshold and the vertical distance is less than the threshold are satisfied (step S37; Yes), the double line determination unit 147 determines that the partition line is a double line (step S38). . Then, the parking support unit 140 ends the determination process.

If the conditions that the horizontal distance is less than the threshold and the vertical distance is less than the threshold are not satisfied (step S37; No), the double line determination unit 147 determines that the partition line is a single line (step S39). Then, the parking support unit 140 ends the determination process.

As described above, when the parking support unit 140 of the present embodiment detects a parking area in the longitudinal direction, that is, the vertical direction, the parking support unit 140 divides the attention area R1 in which the edge point P1 indicating the boundary of the parking area is detected. Set at the front end of the line. Then, the parking support unit 140 determines the parking position based on the front end point P2 detected from the region in which the attention region R1 is set. Therefore, the parking support unit 140 can realize parking support at an appropriate position according to the lane markings. Furthermore, by setting the attention area R1, the parking support unit 140 can limit the search range for searching for the shape of the marking line. That is, since the parking support unit 140 can narrow down the search range, it is possible to suppress an increase in processing load. Further, if there is no edge peak, the parking support unit 140 can determine that the edge is not a marking line, since this indicates that there is no leading edge. Thereby, the parking support unit 140 can reduce the possibility of misrecognizing something that is not a lane marking as a lane marking.

Although the embodiments of the present invention have been illustrated above, the above embodiments and modifications are merely examples, and are not intended to limit the scope of the invention. The embodiments and modifications described above can be implemented in various other forms, and various omissions, substitutions, combinations, and changes can be made without departing from the gist of the invention. Furthermore, the configurations and shapes of each embodiment and each modification can be partially replaced.

DESCRIPTION OF SYMBOLS 1...Vehicle, 2c...End part, 8...Display device, 9...Audio output device, 10...Operation input unit, 11...Monitor device, 12...Display device, 14...ECU, 14a...CPU, 15, 15a, 15b, 15c, 15d...Imaging section, 16, 17...Distance measuring section, 140...Parking support section, 141...Data acquisition section, 142...Vertical line detection section, 143...Attention area setting section, 144...Edge detection section, 145...Attention Area division section, 146... Front end point detection section, 147... Double line determination section, 148... Parking position determination section, P1... Edge point, P11... Black and white edge point, P12... Black and white edge point, P2... Front end point, R1... Attention area, A1...imaging area.

Claims (6)

a first detection unit that detects a partition line that is substantially parallel to the longitudinal direction of the parking area;
a setting unit that sets a region of interest, which is an area in which an edge point indicating a boundary of the partition line is detected, at a position defined by the partition line detected by the first detection unit;
a second detection unit that detects, based on the edge point in the attention area, a front end position indicating an end of the partition line that is on the entry side where a vehicle enters the parking area and is substantially parallel to the longitudinal direction;
a determining unit that determines a parking position based on the front end position;
a determination unit that determines whether the partition line substantially parallel to the longitudinal direction is a double line;
Equipped with
When the determining unit determines that the dividing line substantially parallel to the longitudinal direction is a double line, the setting unit may, based on the dividing line substantially parallel to the longitudinal direction formed by the double line, setting the attention area in a first area within a predetermined distance from the end of the partition line;
The determining unit determines the parking position based on the front end position detected from the first area.
Parking assistance device. further comprising a dividing unit that divides the region of interest into a plurality of divided regions by dividing lines in an orthogonal direction orthogonal to the longitudinal direction,
The second detection unit detects a front end position indicating an end of the partition line, which is on an entry side where a vehicle enters the parking area and is substantially parallel to the longitudinal direction, based on the edge point in the divided area. do,
The parking assist device according to claim 1. The second detection unit specifies the front end position based on the divided area defined based on the edge point and a detection result of a partition line.
The parking assistance device according to claim 2. When the determining unit determines that the dividing line substantially parallel to the longitudinal direction is not a double line, the setting unit configures a second area including an end of the dividing line formed by a single line; setting the attention area in a third area located on the side of the area and in the lateral direction of the parking area;
The determining unit determines the parking position based on the front end position detected from the second area when a partition line in an orthogonal direction perpendicular to the longitudinal direction is not detected from the third area.
The parking assist device according to claim 1 . The determination unit determines the parking position based on the front end position and the line width of the division line when the orthogonal division line is detected from the third area.
The parking assistance device according to claim 4 . The determination unit determines that when a distance in an orthogonal direction perpendicular to the longitudinal direction between a partition line substantially parallel to the longitudinal direction and a partition line adjacent to the partition line substantially parallel to the longitudinal direction is equal to or less than a threshold value, Determine that it is a double line,
A parking assistance device according to any one of claims 1 to 5 .

Images