JPH11339194A - Parking aiding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly aid parking by sensing surrounding situations. SOLUTION: An image recognition device 15 discriminates a parking section 30 based on an image from a camera 17 by using a steering sensor 2 for detecting the steering state of a vehicle, calculates the predicted traveling locus 20 of the vehicle based on information from the sensor 20 and information for aiding parking based on the information of the locus 20 and the section 30 is displayed on a display 13 and a voice message based on the information is generated from a speaker 8 through a voice synthesis circuit 7 to inform a driver of the information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a parking assist device for assisting parking such as parallel parking or garage parking of a vehicle. The present invention relates to a parking assist device that displays a predicted traveling trajectory (also referred to as a taxiway) together with a rear image on a display and assists a parking operation by voice output.

[0002]

2. Description of the Related Art Conventionally, there has been known a method of assisting a driver who is unfamiliar with parking such as parallel parking or garage parking during parking operation. For example, Japanese Patent Application Laid-Open No. 7-1732
In JP-A-8, a CCD camera and a distance sensor for measuring the distance are provided around the vehicle body to detect the surroundings of the vehicle, and a peripheral image around the vehicle is displayed in a bird's-eye view on a display provided in the vehicle interior. Provide the driver with the surrounding situation.

In Japanese Patent Application Laid-Open No. 59-201082, a steering angle is detected by a steering sensor, and a steering operation at the time of a parking operation is performed based on the detected position of the own vehicle and predetermined data relating to parking. One that calculates a steering angle and displays the steering angle and a command angle required for steering on a simple display.
In the gazette disclosed in Japanese Patent Laid-Open Publication No. H10-264, a distance to an obstacle (particularly, a car parked next to a parking space to be parked, etc.) is measured by a distance measuring sensor for object detection provided behind the vehicle, A method of detecting a turning start position based on the maximum steering angle according to the distance and notifying the driver of the turning start position, and, in Japanese Patent Application Laid-Open No. 6-234341, a vehicle using a CCD area sensor or a steering sensor. There is known a method in which a guide path for guiding the vehicle is calculated and a steering angle instruction is given to the driver by a voice message based on the calculated guide path and the current position.

[0004]

However, when the vehicle is parked, the vehicle is steered to a predetermined position with respect to a parking section indicated by a white line or the like so as to be parallel near the center. It is desirable to make the vehicle back, but in any of the above-described conventional techniques, it is necessary to recognize the compartment to be parked itself and give an appropriate instruction to the driver at the time of the parking operation to park the vehicle correctly. In other words, it does not assist parking in an appropriate state.

[0005] The present invention has been made in view of the above-mentioned problems, and has as its technical object to assist in parking properly so as to park at a correct position after identifying the parking section itself.

[0006]

The technical measures taken to solve the above-mentioned problems include a steering state detecting means for detecting a steering state of a vehicle, and a parking section by image recognition based on an image from a camera. Providing the driver with parking section detecting means for identification, predicted driving path calculating means for calculating a predicted driving path of the vehicle based on information from the steering state detecting means, and information assisting parking based on the predicted driving path and information from the parking section. And a notifying means.

By the above technical means, a parking section is detected by image recognition based on an image from a camera, and a predicted traveling trajectory of the vehicle is obtained from information from the steering state detecting means. Then, it is possible to provide information for assisting parking after determining whether or not parking can be performed correctly.

In this case, the parking section can be easily detected by image recognition if the white line constituting the parking section is recognized.

The relative positional relationship between the parking section and the vehicle is determined by the inclination angle θ of the vehicle center axis with respect to the center axis of the parking section.
m, the distance L between the camera position and the center axis of the parking space, and the distance D to the entrance of the parking space, and the relationship between the relative positions of the parking space and the vehicle can be accurately determined.

[0010] Further, the informing means may include at least one of a steering timing, a steering direction, and a steering amount by means of a display and especially voice.
If one of them is notified, it is possible to provide the driver with information on parking while visually confirming the rear.

If the rear image and the predicted travel locus from the camera are superimposed on the display, the surrounding situation can be accurately detected, and the predicted travel locus corresponding to the steering angle can be displayed in the rear image. You can see if it passes.

If the display form of the predicted travel trajectory is varied according to the relative positional relationship with the parking section, the predicted travel trajectory can be changed to a predetermined parking section by changing the display form (display color, etc.) of the predicted travel trajectory. On the other hand, it is easy to determine whether or not parking is possible.

[0013]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a system configuration diagram of the parking assist device 1. In this figure, a controller 16 for controlling the parking assist device 1 includes a CCD camera (hereinafter, referred to as a camera) 17 for photographing the rear of the vehicle, a steering sensor 2 for detecting a steering angle of a steering wheel (hereinafter, referred to as a steering) 21. A shift lever reverse switch 3 for detecting a reverse (reverse) state of a shift lever of the transmission, a parking switch 4 for operating a parking assist function at the time of a parking operation, and a wheel speed sensor 5 for detecting left and right wheel speeds of driven wheels. 6, the controller 16 displays a rear image of the vehicle on the display 13 and a predicted travel locus 20 described later on the basis of these signals.
Can be displayed in an overlapping manner. Further, in this device 1, the speech synthesis output is output from the speaker 8 by the speech synthesis circuit 7.
A voice is issued to the driver to assist the driver in parking.

In the controller 16, C for controlling
PU 11, a graphics drawing circuit 12 for drawing graphics on the display 13, a superimposing circuit 9 for superimposing a graphics signal and a rear image from the camera 17, a synchronization signal is extracted from the camera image and sent to the graphics drawing circuit 12. Supply synchronization separation circuit 10,
A parking section detection image recognition device 15 and the like that receive an image signal from the camera 17 and perform image recognition of the parking section 30 (particularly, the white line 31) are provided.

A steering 21 is provided on the display 13.
A steering angle state display (display marker) 14 whose lighting state changes according to the steering angle state of the vehicle is provided symmetrically. When the steering angle is large, the display marker 14 lights up in a steered direction in a large number of times or is neutral. At this point, only the center marker 14 is illuminated, so that it is possible to know how much the steering wheel 21 is turned together with the rear image. That is, the display marker 14 displays the steering timing, the steering direction, and the steering amount at the time of the parking operation, and can notify the driver.

FIG. 2 is a mounting diagram when the parking assist device 1 is mounted on a vehicle. A camera 17 for imaging the rear is mounted near the upper center of the license plate at the rear of the vehicle, and is installed with the optical axis directed downward. Specifically, FIG.
As shown in FIG. 14, the camera is mounted downward (about 30 degrees) at the center of the rear of the vehicle, and the camera itself secures a field of view of 140 degrees left and right by a wide-angle lens, and can photograph an area up to about 8 m behind. It has become.

A display 13 is provided on a panel surface of a center console in the cabin of the vehicle, and a controller 16 is mounted above the glove box. Further, the parking switch 4 for issuing a request for assisting parking is provided near the center console which is easy for the driver to operate.

Here, the steering sensor 2 will be described with reference to FIG. The steering sensor 2 is a steering angle (steering steering angle) when the steering 21 is turned.
Is to be detected. The slit plate 2a is attached so as to rotate integrally with the steering column shaft 23, and two sets of photo interrupters 2c and 2b having a phase difference of 90 ° are attached. In this configuration, by turning on / off the phototransistor by passing or blocking light by rotating a plurality of slits provided on the disk plate 2a in a circumferential shape, two A-phase and B-phase signal pulses ( (See FIG. 4C). This means that the phase B is delayed by 90 ° or advanced ahead of the phase A depending on the rotation direction of the steering 21. In this case, the output of the steering angle of 1 ° / pulse is used. Used.

Next, the processing of the controller 16 will be described with reference to FIG. The controller 16 starts the program shown in FIG. 5 when the power is turned on (the accessory switch is turned on).

In step S101, various initial values are set in a memory required for this processing. Then, step S10
In step 2, the state of the shift reverse switch 3 is checked. If it is not reverse, the display on the display 13 is stopped in step S115, and the process returns to step S102. On the other hand, when the shift reverse switch 3 is turned on (reverse lever state), step S103 is performed. Step S10
In step 3, the display 13 is switched to the camera image mode to set a mode in which an image behind the vehicle can be displayed as a raw image. That is, since the rear state can be accurately displayed by the raw image, it is possible to confirm a person, an object, or the like crossing the rear.

Next, in step S104, the parking switch 4 for assisting parking during the parking operation is checked. If the parking switch 4 is off (there is no parking assistance request), the graphic screen of the display 13 is cleared in step S112, and only the rear raw image is displayed on the display 13 (described later in FIG. 1). In a state where the predicted traveling locus 20 is not displayed), the process proceeds to step S10.
Return to 2.

On the other hand, if the parking switch 4 is ON (there is a request for parking assistance) in step S104, the process proceeds to step S105. In step S105, a predetermined audio signal is output to the audio synthesizing circuit 7, and the speaker is output. 8 to output audio. That is, according to the situation that the parking operation has been started, "Assist parking. Adjust the trajectory to the desired position and back while paying attention to the surroundings.", "Starting the parking guide now Predetermined voice messages such as "Turn the steering wheel so that the tip of the green (expected running trajectory) display points toward the parking space.", "Be careful on the right (left)." Then, guidance is provided to the driver by voice synthesis. By listening to this voice message, the driver can know that assistance in parking operation has been started.

Next, in step S106, a steering sensor value N is read from the steering sensor 2, and
The turning radius R at the time of the parking operation is calculated based on the value. Specifically, when the reading of the steering sensor 2 is detected at the time of detecting the rising edge of the A-phase signal, an interrupt is generated in the main program, and the interrupt processing shown in FIG. 6 is executed. That is, the state of the B-phase signal is checked in step S201 of FIG. 6, and if the B-phase signal is high (H: high potential), the steering count value N is incremented in step S202; if low (L: low potential), it is decremented. Then, the value is stored in the memory. In this case, the steering count value N is θ = N because one pulse is 1 °. However, since the absolute steering angle of the steering wheel 21 becomes indeterminate only by counting the steering value N shown above, the neutral point of the steering angle is detected by the method shown in FIG. 7 and the neutral point is determined by setting N = 0. .

The determination of the neutral point will now be described with reference to FIG. This process is executed by a timer interrupt having a one-second cycle. Here, the vehicle speed is also normally calculated from signals from the known left and right wheel speed sensors 5 and 6 provided on the wheels. Step S301, step S30
In 2, the signals (pulses) from the left and right wheel speed sensors 5, 6
Is counted by a hardware counter incorporated in the CPU 11 in the controller, and the left and right wheel speeds are read out in this timer interrupt routine, and are stored in the memories NR and NL in which wheel speed sensor values are stored. After reading, the counter itself is cleared, and NR and NL indicate the number of pulses per second.

In the next step S303, the average value (NR + NL) / 2 is calculated from NR and NL, and this value is multiplied by the circumference of the tire, so that the vehicle speed V can be easily obtained by a known method. Next, regarding the reference setting of the steering sensor 2, in steps S304 to S306, when the vehicle speed is equal to or higher than the predetermined speed (10 km / h), the vehicle is driven in a state where there is almost no pulse difference between the left and right wheel speed sensors 5 and 6. Assuming that the vehicle is in the straight traveling state, the steering counter N is reset to zero in step S306, and the neutral point of the steering angle is obtained.

When the steering process is completed, the process returns to the main routine of FIG. 5, and in step S107, the parameters of the predicted traveling locus 20 are calculated. The running locus parameter calculation is for obtaining the turning radius R from the steering angle θ and the wheel base L from the geometric relationship shown in FIG.

Thereafter, in step S108, the image of the camera 17 is fetched into the image recognition device 15, and
At 109, a lane marking detection process is performed.

This lane marking detection process is performed by the method shown in FIG. That is, the image input from the camera 17 is, for example, a monochrome image (640 × 480 dot size), and each point has a gray scale of 0 to 255. From this image, first, in step S401, a white line 31 serving as a feature point of the parking section 30 according to a known technique
Edge detection. Specifically, a method is used in which the edge of the white line 31 is detected by a known Sobel filter, the peak of a portion exceeding an appropriate threshold value is detected, and the edge points forming the boundary are extracted. In this case, the recognition of the white line can easily detect the feature point of the parking section by image recognition.
Next, in step S402, these edge points are converted into road surface coordinates shown in FIG. In this coordinate system, straight line detection is performed in step S403.
If a known Hough transform is used as a method for obtaining this straight line, detection becomes possible.

In the straight line detection by the Hough transform, after detecting the edge of the white line 31 of the parking section 30 captured by the camera 17, a straight line is detected from a set of edge points which are inversely transformed into the (X, Z) plane. Use in case. That is, this is a known technique, and the conversion is as follows, assuming that each edge point is (x, y).

X = x · Hc / (fsin θ−ycos θ) (1) Z = Hc · (ycos θ + fcos θ) / (fsin θ−ycos θ) (2) u = X (3) v = −Z (4) p = ucos θ + vsin θ (5) That is, in order to perform this conversion, first, the expressions (1) and (2) are converted in order to describe each edge point by (X, Z) coordinates.

Next, as a Hough space, (ρ, θ)
And (3) to (5) for each edge (X, Z).
Calculate ρ using the formula. At this time, a curve is drawn on the (ρ, θ) space between θ = −π / 2 and π / 2. In practice, this corresponds to an operation of dividing the ρ, θ space into a grid pattern, correlating the space with a two-dimensional matrix, changing θ, and incrementing each corresponding matrix value.

In this way, when all the candidate edge points are drawn on the matrix, the intersections of the curves concentrate on the candidate line segments, so that the matrix value has a large value. Then, an appropriate matrix position (ρ, θ) is selected according to a certain threshold or the like, and one straight line can be extracted correspondingly.

Then, after detecting the straight line, returning to FIG. 8, in step S404, a candidate line segment constituting the parking section 30 is extracted from the straight line detected by the Hough transform. Specifically, as shown in FIG. 17, Lm: body length, Wm: vehicle width, lm: distance between the rear wheel axle and the camera, R
min: assuming the minimum turning radius of the vehicle, as shown in FIG. 10, the parking section 3 from the center of the rear end of the vehicle (camera position).
Distance D to entrance 0, center of rear end of vehicle (camera position)
Lateral displacement distance (center deviation) L from the parking lot center,
The angle (vehicle angle) θm between the vehicle and the parking space can be obtained from the geometric relationship.

Here, the line segments of C, D, E, and F in FIG. 17 are extracted in the allowable range by comparing the angle and the order of arrangement, and comparing the parking section width W1 and the white line width W2.

Next, in step S405, it is checked whether or not a side line candidate (white line) has been found. If there is no side line candidate, this processing is terminated. A detection process of an orthogonal component orthogonal to the side line is performed. That is, this corresponds to the search for the virtual line segments A and B in FIG.

Next, if there is no orthogonal line segment candidate in step S407, this process is terminated. If there is an orthogonal component candidate, L, θ is determined in step S408.
m and D are calculated and obtained. It should be noted that since a specific calculation method for obtaining L, θm, and D can be calculated based on a geometric relationship from FIG. 17, the calculation method is omitted.

Next, the determination of the state of the vehicle will be described with reference to FIG.

In step S501, it is checked whether or not a parking section has been detected. If no parking section has been detected, it is determined in step S512 that it is unknown, and an unknown flag is set. On the other hand, if the section is detected, the process proceeds to step S502, where the center deviation L, the vehicle body angle θ
From m, a reference turning radius R0 is obtained. This is Figure 1
The parking lot 3 is represented by the (condition 1) shown in FIG.
The radius of a circle that is in contact with both the central axis of 0 and the central axis of the vehicle will be determined. In this case, when the turning center is on the right side in the traveling direction of the vehicle, the radius R0 is positive.

Next, in step S503, the turning radius R0 is compared with Rmin, which is the minimum turning radius of the vehicle. If the turning radius Rmin is smaller than this, the vehicle cannot enter the parking space 30, so it is determined in step S511 that the vehicle cannot be parked. Set the disable flag. Here, if the turning radius R0 is equal to or more than Rmin and parking is possible, the process proceeds to step S504, and FIG.
D shown in FIG. 7 is calculated by the (condition 2) formula. This is to determine whether or not the vehicle can be parallel at an appropriate position in the parking space 30 based on the locus of the radius R0 for turning.
In step S505, it is determined whether parking is possible under the condition of d <Lm-lm. If the condition shown in step S505 is not satisfied, the vehicle does not become parallel when entering the parking section 30.

Next, the process proceeds to step S506, where condition 3 is checked. That is, Rx is obtained by the relational expression shown in condition 3 in FIG. This is to check whether the inside of the turn does not interfere with the corner of the parking section when the vehicle enters the parking section 30. In step S507, Rx
If the condition of || R0 | -Wm / 2 is satisfied, it can be considered that the possibility that the inside of the turning of the vehicle comes into contact with an adjacent vehicle or the like is reduced. However, this is merely a calculation reference, and in practice, it may be considered that an adjacent vehicle runs out of the vehicle. Therefore, if the above conditional expressions (conditions 1 to 3) are satisfied, step S508 is executed.
Proceed to.

In step S508, if the turning radius R based on the current steering angle and the reference radius R0 are substantially the same, then in step S509, parking is enabled with the steering angle as it is, and the parking enable flag is set. set. On the other hand, when the turning radius R does not substantially match the reference radius R0, it is determined that steering correction is necessary in step S510, and the steering correction flag is set. However, as shown in FIG. 20, R0> Rmin
In the case of, there is a degree of freedom, and depending on the situation, steering correction is not required.

Based on this result, step S111 in FIG.
Then, audio output 2 is performed. The audio output 2 here is to output a predetermined text to notify the driver based on the flag set by the state determination in step S110. Specifically, as shown in FIG. 19, if the status is "unknown", "the parking stall has not been detected", and if the status is "impossible", "go ahead and try again." If the parking is possible with the steering angle as it is, a voice message such as "Please pay attention to the surroundings and back as it is" is output from the voice synthesis circuit 7 to the speaker 8. In this case, the designation of the color in the case of displaying the predicted traveling locus 20 on the display 13 is also set to the display color shown in FIG.

Next, graphics redraw processing is performed in step S112. The processing here is to display the predicted travel locus 20 on the display in a manner superimposed on the rear image, and since the road surface coordinates must be converted into camera coordinates in order to perform this display, the following coordinate conversion is performed. I do.

Therefore, this coordinate conversion is performed as shown in FIG. 16, (X, Y, Z): road surface coordinates, (x, y): camera coordinates (CCD element surface), f: lens focal length, (x ′). ,
y ′, z ′): lens coordinates (z ′ coincides with the optical axis), θ
Assuming that c is the camera mounting angle and Hc is the mounting height from the road surface, the following relational expression is established. That is, x = fxx / z '(1') y = fyx / z '(2') x '= X (3') y '= Zsin θc + ( Y−Hc) cos θc (4 ′) z ′ = Zcos θc− (Y−Hc) sin θc (5 ′) From the above, Y = 0 when limited to only the coordinates on the road surface, and x and y are If found, x = f ＝ X / (Zcosθc + Hcsinθc) ・ (6 ′) y = f ・ (Zsinθc−Hccosθc) / (Zcosθc + Hcsinθc) (7 ′) Therefore, when the point (X, Z) on the road surface coordinates is photographed by the camera, the coordinates (x, y) on the graphics screen (camera coordinates) are obtained from the equations (6 ′) and (7 ′), and Can be superimposed on the image.

In this case, x, y obtained by the above method
Is displayed on the display, and various display methods are conceivable as shown in FIG. That is, FIG. 12 (a) shows a method of displaying a predicted track through which the left and right wheels of the vehicle pass, FIG. 12 (b) a method of displaying a traveling area where the vehicle travels when parking, and (3) a constant distance interval (ladder). (Interval: 50cm)
There is a method of displaying like a ladder so that you can understand,
Here, by using (c), a method is adopted in which the sense of distance and the angle of the vehicle body at each position can be easily understood during the parking operation. still,
In this case, the length l of the predicted vehicle trajectory 20 is set to a fixed length (for example, 3 m) or a fixed angle, and the color is changed between a turning state (green) and a straight traveling state (blue). Further, it is also possible to adopt a method in which only the tip of the expected trajectory is displayed so that it can be easily distinguished.

FIG. 18 is an example of a display screen showing the predicted traveling locus 20 on the display 13 and the state of the steering angle, and shows a state in which the predicted traveling locus 20 changes in response to the steering angle. It is. This monitors a raw image behind the vehicle, and overlaps the ladder-like scheduled traveling trajectory 20 according to the steering angle only when the parking switch 4 is turned on (in a state where parking assistance is requested). It is displayed. In this case, the display marker 1 that displays the steering angle state on a part of the display 13 so as to indicate how much the steering wheel 21 is turned by displaying the predicted traveling locus 20 in the rear image.
Since 4 is displayed together, it is possible to know how much the vehicle is actually turning.

In the next step S113, it is determined whether or not the parking switch 4 is turned on (a parking assistance request is made). If the parking switch 4 has not been turned on (there is no parking assistance request), the process returns to step S102, and the processing from step S102 is repeated. On the other hand, if there is a parking assistance request, the shift lever reverse switch 3 is checked in step S114 to determine whether the vehicle is in the reverse state. If it is not in the reverse state, the process returns to step S102. If it is in the reverse state, the same processing from step S106 to step S114 is repeated.

That is, when there is a parking assistance request in the reverse state, the predicted running trajectory is displayed together with the rear image on the display. What is necessary is to hold it in a proper position and back it. If the steering is straightened when the vehicle enters the parking space horizontally and the vehicle is backed up to the end, the vehicle can enter the parking space 30 correctly. Since these are instructed by voice, it is possible to perform an operation while looking back and confirming safety.

[0050]

According to the present invention, a steering state detecting means for detecting a steering state of a vehicle, a parking section detecting means for identifying a parking section by image recognition based on an image from a camera, and information from the steering state detecting means. By providing a predicted driving trajectory calculating means for calculating a predicted driving trajectory of the vehicle, and a notifying means for providing information for assisting parking to the driver based on the predicted driving trajectory and the information from the parking section, it is possible to use a video from the camera. The parking section is detected by image recognition, a predicted traveling trajectory of the vehicle is obtained from information from the steering state detecting means, and it is determined whether or not parking can be performed correctly from the predicted traveling trajectory and information on the parking section, and then the parking position is determined. The information which assists parking appropriately so that it parks in can be provided.

In this case, the parking section can be easily detected by image recognition if the white line constituting the parking section is recognized.

The information for assisting parking uses the relative positional relationship between the parking section and the vehicle, and the inclination angle θm of the vehicle center axis with respect to the center axis of the parking section, the distance L between the camera position and the center axis of the parking section. And the distance D to the entrance of the parking section, and the relative positional relationship between the parking section and the vehicle can be accurately obtained.

Further, the notifying means includes at least one of a steering timing, a steering direction, and a steering amount by means of a display and particularly a voice.
If one of them is notified, it is possible to provide the driver with information on parking while visually confirming the rear.

If the rear view image from the camera and the predicted travel trajectory are superimposed on the display, the surrounding situation can be accurately detected, and the predicted travel trajectory corresponding to the steering angle can be displayed in any position of the rear view image. You can see if it passes.

If the display form of the predicted travel trajectory is varied depending on the positional relationship with the parking section, the predicted travel trajectory determines whether or not it is possible to park in a predetermined parking section by changing the display form of the predicted travel trajectory. Is easier to determine.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of a parking assist device according to an embodiment of the present invention.

FIG. 2 is a mounting diagram when the parking assist device according to the embodiment of the present invention is mounted on a vehicle.

FIG. 3 is a diagram illustrating a detection range of a camera of the parking assist device according to the embodiment of the present invention.

4A and 4B show a steering sensor according to an embodiment of the present invention, wherein FIG. 4A is a plan view of the steering sensor when attached to a steering column shaft, and FIG. 4B shows an outline of a slit plate and a photo interrupter of the steering sensor. FIG. 3C is a diagram showing the outputs of the A and B phases of the steering sensor.

FIG. 5 is a flowchart illustrating processing of a controller according to the embodiment of the present invention.

FIG. 6 is a flowchart illustrating steering sensor signal processing of a controller according to an embodiment of the present invention.

FIG. 7 is a flowchart illustrating a neutral point process of the steering sensor of the controller according to the embodiment of the present invention.

FIG. 8 is a flowchart of a lane marking detection process shown in FIG. 5;

FIG. 9 is a flowchart of the state determination shown in FIG.

FIG. 10 is an explanatory diagram illustrating a positional relationship between a parking section and a vehicle according to an embodiment of the present invention.

FIG. 11 is an explanatory diagram used for calculating a predicted travel trajectory according to an embodiment of the present invention.

FIG. 12 is a diagram showing a display example of a predicted traveling trajectory according to an embodiment of the present invention, wherein (a) is a display based on predicted tracks, (b) is a traveling area belt display for a vehicle width, and (c) is a display. It is a figure showing a ladder display.

FIG. 13 shows graphics display coordinates of a camera and a display according to an embodiment of the present invention.

FIG. 14 is a diagram illustrating an attached state when the camera of the parking assist device according to the embodiment of the present invention is attached to a vehicle.

FIG. 15 is an explanatory diagram showing a straight line detection method by Hough transform in one embodiment of the present invention.

FIG. 16 is an explanatory diagram illustrating a coordinate conversion method of the parking assist device according to the embodiment of the present invention.

FIG. 17 is an explanatory diagram illustrating a relationship between detection of a parking section of the parking assist device and a vehicle position according to the embodiment of the present invention.

FIG. 18 is a display screen on a display of the parking assist device according to the embodiment of the present invention.

FIG. 19 is a diagram showing a guidance message in a voice output of the parking assist device according to the embodiment of the present invention and a display color of a predicted traveling trajectory on the display.

FIG. 20 is a diagram illustrating how a parking assist device enters a parking space according to an embodiment of the present invention.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 parking assist device 2 steering sensor (steering state detecting means) 3 shift lever reverse switch 4 parking switch 5 right wheel speed sensor 6 left wheel speed sensor 8 speaker (notification means) 11 CPU (running prediction calculation means) 13 display (display) 14) Steering angle state display (marker display) 15 Parking section detection image recognition device (parking section detection means) 17 CCD camera (camera) 20 Estimated trajectory 30 Parking section 31 White line

Claims (6)

[Claims] 1. A parking assistance device for photographing the rear of a vehicle with a camera, displaying an image from the camera as a rear image on a display provided in the vehicle, and assisting a driver in a parking operation. Steering state detecting means for detecting a steering state of the vehicle, parking section detecting means for identifying a parking section by image recognition based on an image from the camera, and calculating a traveling trajectory of the vehicle based on information from the steering state detecting means. A parking assist device, comprising: a predicted travel trajectory calculating means to perform the parking; 2. The parking assist device according to claim 1, wherein the parking section recognizes a white line constituting the parking section. 3. The information for assisting parking uses a relative positional relationship between the parking section and the vehicle, the inclination angle θm of the vehicle center axis with respect to the center axis of the parking section, the distance L between the camera position and the center axis of the parking section. The parking assist device according to claim 2, wherein the parking assist device is calculated from a distance D to an entrance of the parking space. 4. The parking assist device according to claim 1, wherein the notifying unit notifies at least one of a steering timing, a steering direction, and a steering amount by a display and a sound. 5. The parking assist device according to claim 4, wherein the rearward image from the camera and the predicted traveling trajectory are displayed on the display unit. 6. The parking assist device according to claim 4, wherein a display form of the predicted traveling trajectory is changed according to a relative positional relationship with the parking section.