JP2006298258A - Parking supporting method and parking supporting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a parking supporting method capable of guiding a vehicle to a parking target area even after the vehicle enters the parking target area and capable of urging attention to a periphery of the vehicle in guiding the vehicle to the parking target area and a parking supporting device. <P>SOLUTION: A navigation device 1 acquires a picture image data G picking up a rear view field of the vehicle from a rear camera 22 provided on the vehicle, and the picture image data G is stored in an RAM 4 in association with a position of the vehicle at each of the picking up points. Additionally, the latest data G is composed free to visually judge with the past picture image data G including a run-off part when the previously set target frame runs off the edge of a screen and it is displayed on a display 12. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a parking assistance method and a parking assistance device.

2. Description of the Related Art Conventionally, as a device for assisting a driving operation when a car is parked, a device attached to the rear end of a vehicle is used to image a peripheral field of view and output the imaged data to a display disposed in a vehicle interior Is known (see, for example, Patent Document 1). Such a rear camera is mounted substantially at the center of the rear end of the vehicle, and its optical axis is fixed in a state where it is inclined downward by a predetermined angle. And this rear camera can image several meters behind from the rear end of the vehicle.
JP 2000-280823 A

  However, when the vehicle starts to enter the parking frame while traveling backward, the parking assist device described above displays a white line and a wheel stopper indicating the parking frame on the display, but the vehicle approaches the end of the white line and the wheel stopper. As a result, the parking frame protrudes from the screen, and eventually the white line and the wheel stopper are not displayed at all. For this reason, as the driver enters the parking frame, it becomes difficult for the driver to grasp the relative position, the relative direction, and the deviation amount between the vehicle and the parking target area. As a result, the impact caused by the contact between the tire and the wheel stopper may increase, the vehicle may be parked obliquely with respect to the white line, or the vehicle may protrude from the parking frame.

  The present invention has been made in view of the above problems, and its purpose is to provide a parking support method and a parking support apparatus that can guide a vehicle into the parking target area even after the vehicle enters the parking target area. Is to provide.

  Moreover, the objective of this invention is providing the parking assistance method and parking assistance apparatus which can call attention to a vehicle periphery, when guiding a vehicle to a parking target area | region.

  In order to solve the above problem, the invention according to claim 1 is a parking support method for supporting a parking operation using a display unit that displays image data and a control unit, and the control unit includes: Image data obtained by imaging the periphery of the vehicle is acquired at each imaging point from an imaging device provided in the vehicle, the acquired image data is stored in image data storage means, and the latest imaged at the current location of the vehicle And displaying the past image data stored in the image data storage means on the display means and notifying that the past image data is being displayed on the display means. And

  The invention described in claim 2 is a display unit that displays image data, an image acquisition unit that acquires image data obtained by imaging the periphery of the vehicle at each imaging point from an imaging device provided in the vehicle, and Display control means for displaying on the display means the image data storage means for storing the image data taken at each imaging point, the latest image data, and the past image data stored in the image data storage means. And a notifying means for notifying that the past image data is displayed on the display means.

According to a third aspect of the present invention, in the parking assist device according to the second aspect, the display control means generates image composition data by combining the latest image data and the past image data. And a means for displaying the image composite data on the display means.

  According to a fourth aspect of the present invention, in the parking assist device according to the second or third aspect, the notifying unit visually recognizes an image based on the latest image data and an image based on the past image data. The gist is that at least one of the image data is processed and output to the display means so as to be discriminated.

  According to a fifth aspect of the present invention, in the parking assist device according to the fourth aspect, the informing means selects at least one of the past color tone, pattern, and brightness of the image data as the latest image data. The gist is to process it differently and output it to the display means.

  The invention according to claim 6 is the parking assist device according to any one of claims 2 to 4, wherein the notification means suggests that the past image data is displayed on the display means. The gist is to output the guidance display to the display means.

  A seventh aspect of the present invention is the parking assist apparatus according to any one of the second to fifth aspects, wherein the informing means is the latest image data or past image data on the screen of the display means. The gist is to output a display that divides the display area.

  According to an eighth aspect of the present invention, in the parking assist device according to any one of the second to sixth aspects, the notification means displays the past image data from a speaker on the display means. The gist of the present invention is to output a guidance sound or guidance voice that suggests.

  According to the first aspect of the present invention, the control means stores the image data acquired at each imaging point in the image data storage means, and the latest image data is stored in the image data storage means so as to be at different points. The past image data is displayed on the display means. Then, the display means notifies that past image data is displayed. Therefore, the driver can recognize that a part of the screen of the display means is a past image. Therefore, the driver pays attention such as actually observing a place corresponding to the past image, Accidents such as collisions can be prevented.

  According to the second aspect of the present invention, the image acquisition means for acquiring the image data at each imaging point, the image data storage means for storing the image data in the image data storage means, and the latest image data are stored in the image data storage. The display control means is stored in the means and is displayed at different points and displayed on the display means together with past image data. Also provided is a notifying means for notifying that the past image data is displayed on the display means. Therefore, the driver can recognize that a part of the screen of the display means is a past image, so pay attention, such as viewing an area corresponding to the past image, and collide with an obstacle. Accidents such as can be prevented in advance.

  According to the third aspect of the present invention, the display control means includes the image composition means, and generates the image composition data obtained by compositing the latest image data and the past image data. A certain image can be displayed on the display means.

  According to the fourth aspect of the present invention, the notifying means can determine at least one of the image data so that the image based on the latest image data and the image based on the past image data can be discriminated by visual recognition. Is processed. For this reason, since it is possible to distinguish which range of the screen corresponds to the image based on the latest or past image data, it is easy to pay attention to the area of the past image.

  According to the fifth aspect of the present invention, at least one of the color tone, pattern, and brightness of the past image data is processed so as to be different from the latest image data, and is output to the display means. For this reason, when the past image data and the latest image data are displayed on the display means, the display area of each image data can be easily determined visually.

  According to the invention described in claim 6, the notifying means outputs a guidance display indicating that past image data is displayed to the display means. For this reason, the driver | operator can recognize that the past image data is displayed by visually recognizing the screen of a display means.

  According to the seventh aspect of the invention, the notification means outputs a display indicating the display area of the latest image data or past image data on the screen of the display means. For this reason, since it is possible to distinguish which range of the entire screen the image based on the latest or past image data can be distinguished, it is possible to easily determine the area to which attention should be paid.

  According to the invention described in claim 8, the notifying means outputs a guide sound or a guide voice suggesting that past image data is displayed from the speaker. For this reason, the driver can know that the past image data is displayed on the display means even when he / she keeps an eye on the display means.

  Hereinafter, an embodiment in which the parking assist device of the present invention is embodied in a navigation device that is mounted on an automobile (vehicle) and performs route guidance will be described with reference to FIGS. FIG. 1 is a block diagram illustrating the configuration of the navigation device 1.

  As shown in FIG. 1, the navigation device 1 includes a control device 2 as a control means. The control device 2 includes a control unit 3 that performs main control, a RAM 4 as an image data storage unit, and a ROM 5. The control unit 3 performs various processes according to various programs such as a route guidance program, an image data storage program, a composite data output program, and a parking guidance program stored in the ROM 5. The control unit 3 constitutes an image acquisition unit, a display control unit, and a notification unit described in the claims.

  The ROM 5 stores contour drawing data 5a. The contour drawing data 5 a is data for displaying the contour of the vehicle C on the display 12. The contour drawing data 5a is set according to the size of the vehicle C, and when the vehicle C is a small vehicle, the contour of the size of the small vehicle according to the screen is drawn.

  In addition, the control device 2 includes a GPS receiver 6 and a vehicle position detector 7. The GPS receiver 6 receives radio waves from GPS satellites. Based on the detection data received from the GPS receiving unit 6, the own vehicle position detection unit 7 periodically calculates positioning data such as latitude, longitude, altitude indicating the absolute position of the own vehicle position.

  Furthermore, the control device 2 includes a sensor interface (I / F unit 8). The control unit 3 acquires the speed and relative direction of the vehicle C from the vehicle speed sensor 20 and the gyro sensor 21 disposed in the vehicle C via the I / F unit 8, and uses these to detect the vehicle position detection unit 7. Send to. The own vehicle position detection unit 7 generates own vehicle position data by autonomous navigation based on the vehicle speed and the relative direction based on the reference own vehicle position calculated from the positioning data by the GPS receiving unit 6. At this time, the host vehicle position detection unit 7 calculates the center point 25 of the axle C2 of the rear wheel C1 of the vehicle C as host vehicle position data, as shown in FIG. And while acquiring positioning data from GPS receiving part 6 sequentially, the own vehicle position by autonomous navigation is corrected and the own vehicle position is determined.

  The control device 2 includes a map data storage unit 9. The map data storage unit 9 stores route data 9a and map drawing data 9b as map data. The route data 9a stores node data indicating nodes indicating intersections and turning points of roads, link data indicating links connecting the nodes, and the like. The control unit 3 performs route guidance processing to a destination. When performing the above, the route is searched according to the route data 9a and the route guidance program stored in the ROM 5. Further, the control unit 3 collates the vehicle position data calculated as described above with the travel locus and the route data 9a, positions the vehicle position coordinates on an appropriate road, and further corrects the vehicle position data. To do. The map drawing data 9b is data for displaying a map from a wide range to a narrow range, and is associated with the route data 9a.

  In addition, the control device 2 includes an imaging data acquisition unit 10 that constitutes an image acquisition unit. The imaging data acquisition unit 10 acquires imaging data from the rear camera 22 as an imaging device provided in the vehicle C for each predetermined movement amount of the vehicle C or for each predetermined time.

  As shown in FIG. 2, the rear camera 22 is attached to the approximate center of the rear end of the vehicle C, such as a back door of the vehicle C, with the optical axis A facing downward. The rear camera 22 includes an optical mechanism including a wide-angle lens, a mirror, and the like, and a CCD image pickup device (none of which is shown). As shown in FIG. 3, the rear camera 22 has a rear visual field of, for example, 140 degrees to the left and right, and can capture an imaging range S (peripheral field of view) about 3 m rearward including the rear end of the vehicle C. Further, the control unit 3 determines the imaging range S of the rear camera 22 based on the own vehicle position data acquired from the own vehicle position detection unit 7 in the XY coordinate system (hereinafter referred to as a vehicle coordinate system) for the vehicle C. Can be computed as This vehicle coordinate system is a coordinate of a horizontal plane parallel to the road surface.

  The imaging data generated by the rear camera 22 is analog / digital converted digital data, and the imaging data acquisition unit 10 provides the control unit 3 as image data G that can be subjected to image processing such as various corrections and synthesis. Send. When the control unit 3 acquires the image data G, as shown in FIG. 4, the control unit 3 temporarily stores the image data G in the RAM 4 in association with the vehicle position data 16 of the point where the image data G (imaging data) is generated. . At this time, the image data G may be stored in association with the imaging range data 17 indicating the coordinates of the imaging range S calculated based on the vehicle position data 16. The own vehicle position data 16 and the imaging range data 17 are indicated by coordinates in the vehicle coordinate system. Further, since the rear camera 22 uses a wide-angle lens, when the image data G is displayed on the display 12 serving as a display unit included in the navigation device 1, so-called distortion aberration occurs in which an image around the screen is shrunk. Yes.

  Furthermore, as shown in FIG. 1, the control device 2 is connected to a steering sensor 23 and a shift sensor 24 provided in the vehicle C via the I / F unit 8. The control unit 3 acquires a steering angle detection signal and a shift position signal from the steering sensor 23 and the shift sensor 24. Then, the control unit 3 determines whether or not the shift position of the transmission is in the reverse state based on the shift position signal. When the shift position is reverse and the image behind the vehicle C is simply displayed, the control unit 3 displays the image data G of the current location temporarily stored in the RAM 4 according to the parking guidance program stored in the ROM 5. The rear view screen 40 as the peripheral view screen shown in FIG. 5 is read and displayed on the display 12. Further, the vehicle width extension line 42 and the expected trajectory line 41 based on the steering angle detection signal are drawn on the rear view screen 40 in a known manner. The expected trajectory line 41 indicates a travel trajectory according to the steering angle and the vehicle width of the vehicle C. The vehicle width extension line 42 is an index obtained by extending the vehicle width of the vehicle C rearward. Therefore, the driver steers according to the amount of deviation between the predicted trajectory line 41 and the vehicle width extension line 42 when parking.

The display 12 of the navigation device 1 is a touch panel, and outputs the image data G transmitted from the control unit 3 and image composite data obtained by combining the image data G at a predetermined timing. Further, when the shift position of the transmission is not reverse, the display 12 outputs the map drawing data 9b transmitted from the control unit 3, and the map screen 12a around the own vehicle position (see FIG. 1). Is displayed along with the vehicle position mark.

  Further, the control device 2 includes an input signal acquisition unit 13, and the input signal acquisition unit 13 pushes the input signal input by the touch panel operation of the display 12 and the operation button 14 provided in the navigation device 1. The input signal input by is acquired and transmitted to the control unit 3. In this operation button 14, a target setting button 14a is provided.

  The target setting button 14a is used to set a parking target area when performing a parking operation. When the target setting button 14a is operated at a point where the parking area R defined by the white line 43 marked on the road surface is behind the vehicle C as shown in FIG. 13, for example, as shown in FIG. A target setting screen 44 is displayed on the display 12. On the target setting screen 44, a background image 45 and a substantially trapezoidal parking target frame (hereinafter referred to as a target frame 46) superimposed on the background image 45 are displayed. A target frame 46 as a parking target area and a setting index is an index for setting a parking target area, and has a predetermined shape and size on the initial screen. When the enlargement button 47 and the reduction button 48 displayed on the target setting screen 44 are input, the target frame 46 is enlarged or reduced. As shown in the target setting screen 44 of FIG. (Shape) can be changed along the white line 49 in the background image 45. Further, the target frame 46 can be slid and the position of the target frame 46 can be set by a touch panel operation or the like. Then, by operating the setting button 56, the size (shape) and position of the target frame 46 are determined.

  As shown in FIG. 1, the navigation device 1 includes a speaker 15. The speaker 15 outputs various guidance voices and guidance sounds based on the voice output signal transmitted from the control unit 3.

  Further, as shown in FIG. 1, the control device 2 includes an image processing unit 11 that constitutes an image synthesis unit and a notification unit. The image processing unit 11 acquires the image data G from the RAM 4 and corrects the image data G, generates image synthesis data, and the like. In the correction of the image data G, the image processing unit 11 corrects distortion aberration caused by the wide-angle lens that has occurred in each image data G.

  Further, as shown in FIG. 6, the image processing unit 11 has coordinate points set on an xy coordinate system (hereinafter referred to as a screen coordinate system) of the imaging screen 30 of the rear camera 22 at the vehicle position A, that is, pixel points. Pixel coordinates 31 indicating the coordinates of the vehicle position B are converted into coordinates in the imaging screen 30 of the vehicle position B. At this time, for example, the control unit 3 obtains the moving amount and moving direction of the vehicle C from the own vehicle position A to the own vehicle position B. Then, the image processing unit 11 causes the pixel coordinates 31 in the own vehicle position A to be translated, rotationally moved, and the like based on the moving distance and moving direction, so that the pixel coordinates 33 in the imaging screen 30 of the own vehicle position B are obtained. Convert to

  Alternatively, for example, based on the angle of the optical axis A of the rear camera 22 and the imaging range S, a relational expression between the screen coordinate system (xy coordinate system) and the vehicle coordinate system (XY coordinate system) is obtained in advance. Based on this, the pixel coordinates 31 are converted into coordinates in the vehicle coordinate system. At this time, by adding the coordinates of the center point 25 indicating the vehicle position of the vehicle coordinate system, the pixel coordinates 31 in the imaging screen 30 at the specific vehicle position (own vehicle position A) are converted into the vehicle coordinates. Convert to system coordinate point 32. Further, the coordinate point 32 of the vehicle system coordinate is converted into the pixel coordinate 33 in the imaging screen 30 of the vehicle position B by the above relational expression. As a result, the image data G generated at the vehicle position A can be converted as image data G captured from the camera viewpoint of the rear camera 22 at the vehicle position B.

  Further, when the target frame 46 is set on the target setting screen 44 illustrated in FIG. 14, the image processing unit 11 acquires the coordinates of the screen coordinate system of the target frame 46. Specifically, the image processing unit 11 receives an input signal from the input signal acquisition unit 13 via the control unit 3, and the target frame 46 of the target frame 46 is displayed within the imaging screen 30 of the display 12 as illustrated in FIG. 7. The coordinates of pixel coordinates 31a to 31d corresponding to the four points at the corners are acquired. The display area of the display 12 is an area in the screen coordinate system.

  In addition, the image processing unit 11 acquires a movement amount such as a movement distance and a movement direction from the reference position of the vehicle C from the control unit 3, and displaces the pixel coordinates 31a to 31d based on the movement amount. For example, when the vehicle C is translated from the initial position in a direction approaching the parking area R, the pixel coordinates 31a to 31d are translated in different directions as shown by one chain line in FIG. The target frame 46 is expanded. Further, when the vehicle C turns from the initial position, as shown by the other chain line in FIG. 7, the pixel coordinates 31 a to 31 d are rotated and the shape and position of the target frame 46 are changed.

  In addition, the image processing unit 11 generates and accumulates image data G1 at the current vehicle position (hereinafter referred to as the current location) out of the image data G accumulated in the RAM 4 and the image data G1 in the past. The past image data G2 thus synthesized is synthesized. First, as described above, the image processing unit 11 changes the target frame 46 according to the moving distance and moving direction of the vehicle C, and the changed target frame 46 is displayed on the imaging screen 30 (display 12 on the display 12) of the screen coordinate system. It is determined whether it is accommodated in (region). Then, as shown in FIG. 8, when it is determined that the target frame 46 protrudes from the imaging screen 30, the coordinates of the outside-screen region 51 as the complement portion including the protruding portion 46 a are acquired. Further, as shown in FIG. 14, the rear camera 22 images the rear end portion of the vehicle body such as the rear bumper B (see FIG. 2) of the vehicle C, and therefore the rear end of the vehicle corresponding to the rear end image 54 of the vehicle. The coordinates of the range 55 are acquired and included in the off-screen area 51. The vehicle rear end range 55 is always a constant data area.

  Furthermore, each pixel coordinate of the off-screen area 51 is coordinate-converted from each pixel coordinate in the screen coordinate system to each coordinate point in the vehicle coordinate system, as described above. Then, data of each coordinate point in the vehicle coordinate system is transmitted to the control unit 3. The control unit 3 searches the image data G stored in the RAM 4 for image data G that includes the coordinate points of the vehicle coordinate system in the imaging range data 17. When the corresponding past image data G2 is detected, the past image data G2 is read out. At this time, a plurality of image data G partially including the respective coordinate points may be read out.

  The image processing unit 11 acquires the actual image data G1 at the current location from the RAM 4 via the control unit 3. Further, the target frame 46 shown in FIG. 8 is reduced to fit within the imaging screen 30 as shown in FIG. Then, based on the reduction rate at that time, the image data G1 at the current point is reduced to the reduction range 52 shown in FIG. 9, and reduced image data G1a is generated.

  For example, when one piece of past image data G2 is used, the reduced image data G1a is placed at a relative position between the reduced range 52 and the imaging screen 30 shown in FIG. Based on the past image data G2 generated in the past, the image synthesis data GA is generated. Thereby, the area other than the reduction range 52 in the imaging screen 30 of FIG. 9 is synthesized with the past image data G2. That is, the past image data G2 supplements the complement range 53 (see FIG. 9) corresponding to the off-screen region 51 in FIG. 8, and the image composite data GA that can accommodate the target frame 46 in the imaging screen 30 is generated. The Further, when the image composite data GA is generated using a plurality of past image data G2, different areas of the complement range 53 are complemented by the plurality of past image data G2, respectively.

Next, the processing procedure of this embodiment is demonstrated according to FIG.11 and FIG.12. FIG. 11 shows the procedure of the image data storage process, and FIG. 12 shows the procedure of the image synthesis data output process.
The control device 2 performs the image data storage process and the image composite data output process in parallel.

  The image data saving process shown in FIG. 11 is a process performed by the control device 2 according to the image data saving program stored in the ROM 5. First, the control unit 3 determines whether or not the shift position of the transmission is reverse (step S1-1). Specifically, the control unit 3 makes a determination by acquiring a shift position signal from the shift sensor 24 via the I / F unit 8.

  If the shift position is not reverse (NO in step S1-1), the determination process in step S1-1 is repeated. When it is determined that the shift position is reverse (YES in step S1-1), the control unit 3 acquires the vehicle position data 16 from the vehicle position detection unit 7 (step S1-2). When the own vehicle position data 16 is acquired, the control unit 3 acquires the image data G from the imaging data acquisition unit 10 (step S1-3). And the imaging range data 17 which shows the coordinate of the imaging range S of the present location is calculated, the own vehicle position data 16 and the imaging range data 17, and the acquired image data G are linked | related, and it stores in RAM4 (step S1-). 4).

  Then, the control unit 3 determines whether the I / F unit 8 and the input signal acquisition unit 13 have an end trigger (step S1-5), and when there is a preset end trigger (step S1- 5), the process is terminated. Specifically, the process ends when a shift position signal indicating parking or driving or an ignition module off signal from a vehicle ECU (not shown) is received via the I / F unit 8. Alternatively, when the input operation signal of the target setting button 14a is received via the input signal acquisition unit 13, the process is terminated. If there is no end trigger (NO in step S1-5), the process returns to step S1-2 and the above-described processing is repeated.

  Such processing of step S1-2 to step S1-5 is repeatedly performed for each imaging point where the vehicle position has moved by several tens of mm or when several tens of milliseconds to one second have elapsed. For this reason, when the vehicle C travels backward, the RAM 4 accumulates the image data G that are continuous with each other. The image data G is used not only for the synthesis of the image data G but also when the rear view screen 40 shown in FIG.

  Next, the image synthesis data output process will be described with reference to FIG. This image data synthesis process is a process performed by the control device 2 in accordance with a synthesis data output program stored in the ROM 5.

  First, similarly to the image data storage process, the control unit 3 determines whether or not the shift position is reverse (step S2-1). If the shift position is reverse (YES in step S2-1), the process waits for setting of the target frame 46 (step S2-2). Specifically, the control unit 3 waits for an input operation of the target setting button 14a. When the input signal is acquired from the input signal acquisition unit 13, the target setting screen 44 shown in FIG. 14 is displayed on the display 12. Furthermore, when the target frame 46 is reduced or enlarged on the target setting screen 44 and the setting button 56 is input (YES in step S2-2), the coordinates of the target frame 46 in the screen coordinate system are changed as described above. Acquire and go to step S2-3.

  While the target setting button 14a is not operated or the setting of the target frame 46 is not completed (NO in step S2-2), the rear view screen 40 and the target setting screen 44 shown in FIG. While waiting for the setting of the target frame 46 on the target setting screen 44 in step S2-2.

  In step S <b> 2-3, the control unit 3 acquires the vehicle position data 16. The own vehicle position data 16 when the target frame 46 is set is set as an initial position. For example, when the target frame 46 is set at the position of the vehicle C shown in FIG. 13, that position is set as the initial position.

And the control part 3 calculates the moving amount | distance of the vehicle C (step S2-4). At this time, for example, the control unit 3 calculates a movement amount (movement distance and movement direction) from the initial position.
Further, the control unit 3 transmits the movement amount to the image processing unit 11, and updates the coordinates of the target frame 46 according to the movement amount of the vehicle C as described above (step S2-5). At this time, if the movement amount is equal to or less than the predetermined value, the image processing unit 11 may not update the target frame 46.

Then, based on the image processing unit 11, the control unit 3 determines whether or not the updated coordinates of the target frame 46 protrude from the imaging screen 30 as shown in FIG. 8 (step S2-6).
When it is determined that the target frame 46 is accommodated in the imaging screen 30 (NO in step S2-6), the image data G at the current location is read from the RAM 4, and the rear view screen 40 displaying the rear of the vehicle C is displayed. , The process proceeds to step S2-13 to determine whether there is an end trigger. In the determination of the end trigger, as in the image data storage process, whether or not a shift position signal indicating parking or driving, an off signal of an ignition module from a vehicle ECU (not shown), or whether or not an input operation signal is received by operating the target setting button 14a based on. If there is no end trigger (NO in step S2-13), the process returns to step S2-3.

  When it is determined that the target frame 46 protrudes from the imaging screen 30 (YES in step S2-6), the control unit 3 and the image processing unit 11 estimate necessary image data G based on the amount of protrusion (step S2). -7). As described above, the image processing unit 11 acquires the coordinates of the off-screen region 51 including the vehicle rear end range 55 corresponding to the image of the protruding portion 46a of the target frame 46, the rear bumper B, and the like. Convert coordinates to coordinate system. Then, the coordinate data is transmitted to the control unit 3, and the control unit 3 searches the image data G including the coordinate data in the imaging range data 17. That is, the control unit 3 searches the image data G obtained by imaging the area in order to complement the protruding portion 46a of the target frame 46 and the like.

  When the corresponding past image data G2 is detected from each image data G, the control unit 3 reads the past image data G2 (step S2-8). At this time, for example, the control unit 3 reads one piece of past image data G2 obtained by imaging the imaging range S as shown in FIG. The past image data G2 includes the entire imaging range corresponding to the out-of-screen area 51. Also, the vehicle position data 16 and the imaging range data 17 associated with the past image data G2 are read out.

  Then, the control unit 3 sends the past image data G2, the own vehicle position data 16 and the imaging range data 17 to the image processing unit 11, and the image processing unit 11 corrects the past image data G2 (step S2-9). ). The image processing unit 11 first corrects the distortion aberration of the past image data G2. Then, as shown in FIG. 6, the image processing unit 11 converts the pixel coordinates 31 at the own vehicle position (own vehicle position A) corresponding to the past image data G2 to the current location (own vehicle) by the method described above. The coordinates are converted into coordinates in the imaging screen 30 at position B). Thereby, it is possible to convert the past image data G2 generated at another point as if it was captured from the camera viewpoint of the rear camera 22 at the current point.

Further, the image processing unit 11 performs image processing such as color tone correction and increase / decrease color on the past image data G2 to make it different from the color tone of the image data G1 at the current location. Thereby, when displayed on the display 12, the image of the past image data G2 and the image of the image data G1 at the current location can be distinguished by visual recognition.

  In addition, the image processing unit 11 corrects the image data G1 at the current location (step S2-10). More specifically, the image processing unit 11 reduces the target frame 46 that protrudes from the imaging screen 30 as shown in FIG. 8 so as to fit within the imaging screen 30 as shown in FIG. Then, based on the reduction rate at that time, the image data G1 at the current point is reduced to the reduction range 52 as shown in FIG. 9, and reduced image data G1a is generated.

  Then, the image processing unit 11 combines the reduced image data G1a and the past image data G2 (step S2-11). For example, when one piece of past image data G2 is used, the image processing unit 11 displays the reduced image data G1a by superimposing the reduced image data G1a at the current location on the past image data G2, as shown in FIG. On the other side. At this time, the past image data G2 is overlapped with the reduced image data G1a so as to complement the complement range 53. As a result, image synthesis data GA is generated. When a plurality of past image data G2 is used, the complement range 53 is overlapped with the reduced image data G1a so as to be complemented by the plurality of past image data G2.

  When the image composition data GA is generated, the control unit 3 displays the image composition data GA and other indexes on the display 12 (step S2-12). In a state where only the image synthesis data GA is output, a synthesis screen 58 as shown in FIG. 17 is displayed on the display 12. On the composite screen 58, a current image 59 based on the current reduced image data G1a, a complementary image 60 supplemented with the past image data G2, and a reduced target frame 46 are displayed in an overlapping manner. Since the complementary image 60 is obtained by processing the past image data G2, it can be distinguished from the current image 59 by visual recognition. Therefore, since the current image 59 can be recognized as the image currently captured, even if an obstacle 62 (see FIG. 17) enters the imaging range S of the rear camera 22 at the current location, From the image 59, it can be determined that the obstacle has entered at the present time. Further, since the complementary image 60 can be recognized as a past image, it is possible to pay particular attention to obstacles in the region, such as by visually observing the actual location corresponding to the complementary image 60.

  As shown in FIG. 18, the image processing unit 11 actually uses the contour drawing data 5a (see FIG. 1) stored in the ROM 5 in a region below the vehicle body in the composite screen 58 in FIG. A vehicle outline 61 as a vehicle position index is drawn in an overlapping manner.

  The vehicle outline 61 drawn by the outline drawing data 5a is drawn only in a necessary portion at a position corresponding to the vehicle position in the imaging screen 30. In the composite screen 58a shown in FIG. 18, since the region below the vehicle body, that is, the region supplemented by the complementary image 60 is relatively small, the vehicle contour corresponding to the rear of the vehicle body (for example, around the bumper, back door, and trunk). The line 61 is drawn so as to overlap the portion of the complementary image 60 (past image data G2). As a result, the complementary image 60 below the vehicle body appears to be displayed on the display 12 through the vehicle body. The driver visually recognizes the vehicle outline 61 and the entire target frame 46 to grasp the relative position and displacement amount with respect to the target frame 46 and grasp the relative distance between the vehicle C and the obstacle 62.

  Further, the control unit 3 receives the steering angle detection signal from the steering sensor 23, and displays the expected trajectory line 41 and the vehicle width extension line 42 in the image composition data GA. Here, since steering is not performed, the vehicle width extension line 42 is displayed so as to overlap the expected trajectory line 41.

Next, the control unit 3 determines whether or not there is an end trigger (step S2-13). If there is an end trigger (YES in step S2-13), the process ends as it is. If it is determined that there is no end trigger (NO in step S2-13), the process returns to step S2-3. Then, the above process is repeated.

  When the above process is repeated several times and about half of the vehicle body enters the parking area R, a composite screen 63 as shown in FIG. 19 is displayed (step S2-12). The composition screen 63 displays a current image 64, a complementary image 65, and a vehicle outline 61. In the composite screen 63, the vehicle outline 61 is composed of an outline drawing line 66 indicating the outer shape of the vehicle body and a rear wheel drawing line 67 indicating each rear wheel. That is, as the vehicle C travels back toward the parking area R, the ratio of the complementary image 65 increases in the imaging screen 30, and the display range (reduction range 52) of the current image 59 that is the current image is narrowed. Go. And the ratio of the display range (complementary range 53) of the complementary image 65 increases, and at the same time, the drawing range of the vehicle body indicated by the vehicle outline 61 expands in the direction of the arrow y (the traveling direction of the vehicle C) in the figure, The entire vehicle body is gradually displayed by the vehicle outline 61. In the actual synthesis screen 63, an expected trajectory line 41 and a vehicle width extension line 42 are also displayed, but are not shown here.

  Furthermore, when the entire vehicle body enters the parking area R (target frame 46), a composite screen 68 as shown in FIG. 20 is displayed on the display 12 (step S2-12). On the composite screen 68, a current image 69, a complementary image 70, and a vehicle outline 61 are displayed. As the vehicle outline 61, a front wheel drawing line 71 is displayed in addition to the outer drawing line 66 and the rear wheel drawing line 67. For this reason, it can be confirmed by this vehicle outline 61 whether the vehicle body is straight along the target frame 46. Thus, when the vehicle outline 61 is stored in the target frame 46, the driver performs a shift change, engine stop, and operation of the target setting button 14a, and the control unit 3 determines that there is an end trigger (step) S2-13) The image synthesis data output process is terminated.

According to the above embodiment, the following effects can be obtained.
(1) In the above embodiment, the control device 2 of the navigation device 1 acquires the image data G generated by the rear camera 22 of the vehicle C at each imaging point. Further, the control unit 3 of the control device 2 stores the acquired image data G in the RAM 4 in association with the own vehicle position data 16 and the imaging range data 17 acquired from the own vehicle position detection unit 7. Further, the coordinates of the parking area R (target frame 46) set in advance on the target setting screen 44 are converted by the control unit 3 and the image processing unit 11 in accordance with the position of the vehicle at the current location. Further, the image processing unit 11 determines whether or not the target frame 46 converted according to the position of the vehicle C protrudes from the imaging screen 30. The coordinates of the area 51) are calculated. In addition, the image processing unit 11 and the control unit 3 read out the past image data G2 associated with the coordinates of the off-screen area 51 out of the image data G stored in the RAM 4, and generate the current image at the current point using the past image data G2. By complementing the image data G <b> 1, the image composition data GA in which the target frame 46 is included in the imaging screen 30 is generated. The control unit 3 displays the image composition data GA on the display 12.

  Therefore, for example, even when the vehicle C enters the parking area R and the rear camera 22 cannot capture the entire parking area R, the past image data G2 supplements the image corresponding to the portion where the target frame 46 protrudes from the screen. Thus, the entire target frame 46 can be displayed on the display 12. Accordingly, the driver visually recognizes the entire parking area R (the entire target frame 46) on the display 12 even when the vehicle C enters the parking area R and the parking area R is positioned below the vehicle body. be able to. For this reason, the vehicle C can be parked so that the vehicle body does not deviate or bend from the parking area R.

(2) In the above embodiment, when the image composite data GA is displayed on the display 12, an image based on the image data G1 generated at the current location and an image based on the past image data G2 can be discriminated by visual recognition. Thus, the image is processed to notify (warn) that the past image is displayed. For this reason, it is possible to distinguish between the supplemented area and the area of the image at the current point in the combined screens 58a, 63, and 68, which corresponds to the driver, for example, a past image such as the side of the vehicle C. You can call attention to the place. Therefore, the driver can visually observe the place corresponding to the past image, pay attention, and prevent an accident such as a collision between the obstacle and the vehicle C in advance.

  (3) In the embodiment described above, the control unit 3 and the image processing unit 11 display the target frame 46 for setting the parking area R on the target setting screen 44 on the display 12. Furthermore, the coordinates of the screen coordinate system of the parking area R (target frame 46) are determined based on an input operation for designating the position, shape, or size of the target frame 46 by the driver. For this reason, the driver can set the parking area R at any place on the screen.

  (4) In the above embodiment, the control unit 3 and the image processing unit 11 superimpose the image composition data GA on the composition screens 58a, 63, and 68 in accordance with the vehicle contour line 61 corresponding to the current vehicle position. I did it. For this reason, since it is possible to easily grasp the amount of deviation between the position of the vehicle C and the parking area R (target frame 46), the vehicle C can be more reliably guided into the parking area R.

  (5) In the above embodiment, the image processing unit 11 reduces the image data G1 generated at the current point according to the reduction rate of the target frame 46 and the complement range 53 that the past image data G2 complements, and the past image The data G2 is combined with an area other than the complementary range 53. For this reason, when the composite screens 58a, 63, and 68 are displayed on the display 12, the continuity of images can be improved.

  (6) In the above embodiment, the imaging data acquisition unit 10 acquires the image data G obtained by imaging the imaging range S behind the vehicle C from the rear camera 22 provided at the rear end of the vehicle C. . For this reason, when the vehicle C backs toward the parking area R, the image synthesis data GA is generated using the image data G obtained by imaging the rear of the vehicle C, and parking assistance is performed when the driving operation is difficult. Especially effective.

  (7) In the above embodiment, the image processing unit 11 corrects the distortion aberration of each image data G. For this reason, when the image data G are combined and displayed on the display 12, the continuity of the images can be improved, and the user can be prevented from feeling uncomfortable.

  (8) In the above embodiment, each past image data G2 read from the RAM 4 is converted into the screen coordinate system of the imaging screen 30 at the current location. For this reason, image data generated by imaging from different camera viewpoints can be converted as image data G1 imaged from the camera viewpoint at the current location, so that when displayed as the composite screen 58a, 63, 68, continuous images are displayed. The property can be made better.

  (9) In the above embodiment, since the parking assistance device is embodied in the navigation device 1 including the own vehicle position detection unit 7, the image data G can be stored and its data can be stored using a highly accurate own vehicle position detection function. The image data G can be synthesized. For this reason, it is possible to display the composite screens 58a, 63, and 68 with good image continuity on the display 12 by effectively using the vehicle position data 16.

In addition, you may change each said embodiment as follows.
In the above embodiment, only the case where the vehicle C is parked in parallel has been described. However, when the target frame 46 protrudes from the imaging screen 30 in the case of parallel parking, the current image data is used using the past image data G2. G1 may be complemented.

The target setting button 14a of the above embodiment may be an operation button displayed on the display 12 when the shift position is reversed.
In the above embodiment, the control device 2 receives the input operation signal of the target setting button 14a or the initial position where the shift position is reversed during the image data storage process and the image synthesis data output process. As a zero point), the relative position from the reference point may be detected as the vehicle position only by the vehicle speed and direction.

  In the above embodiment, the parking target area is set by reducing or enlarging the target frame 46 in the initial state, but the target frame 46 is determined by determining any four points on the target setting screen 44. May be generated.

  In the above embodiment, the parking target area is set by the target frame 46, but the white line 43 on the road surface is detected using the image processing of the image data G or a sensor that detects the white line 43, and the control device 2 detects A predetermined distance (for example, 15 cm on the inner side) from the white line 43 may be automatically set as the parking target area.

  In the above embodiment, the target frame 46 set on the target setting screen 44 is changed in the pixel coordinates of the target frame 46 based on the movement amount of the vehicle C, and the changed target frame 46 is captured in the screen coordinate system. It was determined whether or not it was accommodated in the screen 30 (display area of the display 12). In addition to this, every time the vehicle C moves a predetermined distance or every predetermined time, the white line 43 on the road surface is detected and the target frame 46 is set, and the pixel coordinates of the set target frame 46 are It may be determined whether or not it is accommodated in the imaging screen 30 (display area of the display 12) in the screen coordinate system. In this case, coordinate conversion processing based on the movement amount can be made unnecessary.

  In the above embodiment, when the vehicle C moves backward, the target frame 46 is displayed on the screen of the display 12. However, the target frame 46 is not displayed on the display 12, and the coordinates of the target frame 46 acquired in advance are You may make it judge whether it protrudes from the imaging screen 30. FIG. At this time, the target frame 46 is displayed only on the target setting screen 44 and its position and the like can be set by the user. When the vehicle C starts to move backward or moves a predetermined distance, the target frame 46 is displayed on the screen. You may make it not display. Further, as described above, the control device 2 may detect the white line 43 and automatically set the target frame 46. In this way, when various indicators such as the vehicle width extension line 42, the predicted trajectory line 41, and the vehicle contour line 61 are displayed on the screen, it is possible to suppress the screen from becoming complicated and make the screen easier to see. .

  In the above embodiment, the imaging range data 17 is calculated in advance based on the vehicle position data 16, and the imaging range data 17 is stored in the RAM 4 in association with the image data G. However, the imaging range data 17 is not calculated in advance. The image data G only needs to be associated with at least the vehicle position data 16. Then, when reading past past image data G2 to be synthesized, based on the own vehicle position data 16, it may be calculated whether the image data G includes the data of the protruding portion of the target frame 46.

  In the above embodiment, the image data G is not stored in the RAM 4 with the data structure as shown in FIG. 4, but a table storing attribute data including the identifier of the image data G and the vehicle position data 16 is generated. The table may be stored in the RAM 4.

In the above embodiment, when all the drawing lines indicating the size of the vehicle body such as the vehicle outline 61 are accommodated in the target frame 46, the control unit 3 or the image processing unit 11 has an end trigger (step S2- 13 may be determined as YES).

  In the above embodiment, as the vehicle position index, in addition to the vehicle contour line 61, a vehicle occupation area indicating an area occupied by the vehicle C in the screen, a center position of the rear axle, a center point of the center of the vehicle, and the like are used. It may be a point to show.

In the above embodiment, the image processing unit 11 may omit correction of distortion.
In the above embodiment, the vehicle position is the center point 25 of the rear axle, but may be a predetermined position of the other vehicle C such as the center of gravity position.

  In the above embodiment, the coordinate conversion for converting the image data G generated at a point different from the current point to the coordinates of the screen coordinate system of the imaging screen 30 at the current point may be omitted. In this way, the burden on the control device 2 can be reduced.

  In the above embodiment, the past image data G2 is processed so that an image based on the image data G1 generated at the current point and an image based on the past image data G2 can be discriminated by visual recognition. In addition to this, as a notification (warning) indicating that a past image based on the past image data G2 is displayed on the display 12, a character display as a guidance display indicating that the past image is displayed, An icon as a guidance display may be displayed on the composite screen 58 in an overlapping manner. Further, a boundary line as a display indicating the boundary between the current image 59 and the complementary image 60 may be displayed. In this case, since the display area of the current image 59 and the complementary image 60 can be distinguished, the driver can pay attention to the complementary image 60. Further, a guidance sound or guidance sound suggesting that a past image is displayed on the display 12 may be output from the speaker 15. In this case, the driver can notice that the complementary image 60 is displayed even when the user is looking away from the display 12 or not aware of the icon or the like.

  In the above embodiment, when the target frame 46 whose coordinates have been converted with the movement of the vehicle C protrudes from the imaging screen 30 (display area of the display 12), the target frame 46 protrudes from the past image data G2. The image synthesis data GA was generated by complementing the portion. In addition to this, when the vehicle C moves a predetermined distance, image composite data in which past image data having an area below the floor of the vehicle body is arranged in parallel with the current image data may be generated. Good. Then, a notification (warning) suggesting that the past image based on the past image data is displayed on the screen on which such image composition data is output may be performed. This notification (warning) is performed by processing at least one of the color tone, pattern, and brightness of past image data so as to be different from the current image data, or by displaying an icon, a screen display of a line indicating the boundary between the past image and the current image, This is performed by outputting a guidance sound or voice from the speaker 15. Moreover, the resolution of the vehicle outline 61 superimposed on the complementary image 65 in the composite screen 63 may be lowered and displayed so as to blur the vehicle outline.

  The rear camera 22 may be provided at any position on the rear end of the vehicle C. Further, the rear camera 22 may be attached to the front end, the side end, or the passenger compartment of the vehicle C. Also in this case, since the area below the vehicle body can be displayed using the image data G generated by this camera, it is possible to make it easier to visually recognize an obstacle that is difficult to see in front of the vehicle C. In addition, the number of cameras that capture the periphery of the vehicle is not limited to one.

  In the above embodiment, the parking assistance device is embodied as the navigation device 1 that performs route guidance, but may be a device that omits route guidance. In this case, the map data storage unit 9 and the speaker 15 are omitted.

The block diagram explaining the structure of the navigation apparatus of this embodiment. Explanatory drawing explaining the attachment position of the rear camera arrange | positioned at the vehicle. Explanatory drawing explaining the imaging range of the back camera. Explanatory drawing explaining the data structure of the image data accumulate | stored in RAM. Explanatory drawing of the screen displayed on the display of the navigation apparatus. Explanatory drawing of the coordinate transformation process of this embodiment. Explanatory drawing of the update process of the target frame of this embodiment. Explanatory drawing explaining the relationship between the updated target frame and an imaging screen. Explanatory drawing explaining the relationship between the reduced target frame, an imaging screen, and the range complemented. The conceptual diagram of a synthesis | combination with reduced image data and past image data. Explanatory drawing of the process sequence of the image data preservation | save process of this embodiment. Explanatory drawing of the process sequence of the image synthetic data output process of this embodiment. Explanatory drawing explaining the vehicle position which performs an imaging or target setting. Explanatory drawing of the initial state of a target setting screen. Explanatory drawing at the time of the setting of a target setting screen. Explanatory drawing of the past image data containing an area outside a screen. Explanatory drawing of the synthetic | combination screen of the state which just overlapped the reduced image data and the past image data. Explanatory drawing of the composite screen displayed on a display. Explanatory drawing of the synthetic | combination screen at the time of the approach start to a parking target area | region. Explanatory drawing of the synthetic | combination screen at the time of the completion of approach to a parking target area | region.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Navigation apparatus as parking assistance apparatus, 3 ... Image acquisition means, control part as display control means and notification means, 4 ... RAM as image data storage means, 10 ... Imaging data acquisition part which comprises image acquisition means, DESCRIPTION OF SYMBOLS 11 ... Image processing part which comprises an image synthetic | combination means and alerting | reporting means, 12 ... Display as display means, 15 ... Speaker, 22 ... Rear camera as an imaging device, C ... Vehicle, G, G1, G2 ... Image data, G2 ... past image data, GA ... image composition data.

Claims (8)

A parking support method for supporting a parking operation using display means for displaying image data and a control means,
The control means is
From the imaging device provided in the vehicle, obtain image data obtained by imaging the periphery of the vehicle at each imaging point,
The acquired image data is stored in image data storage means,
The latest image data captured at the current location of the vehicle and the past image data stored in the image data storage means are displayed on the display means,
A parking assistance method characterized by notifying that the past image data is displayed on the display means. Display means for displaying image data;
Image acquisition means for acquiring image data obtained by imaging the periphery of the vehicle at each imaging point from an imaging device provided in the vehicle;
Image data storage means for storing the image data captured at each imaging point;
Display control means for displaying the latest image data and the past image data stored in the image data storage means on the display means;
A parking assistance device comprising: an informing means for informing that the past image data is displayed on the display means. In the parking assistance device according to claim 2,
The display control means includes image composition means for generating image composition data obtained by combining the latest image data and the past image data, and displays the image composition data on the display means. Parking assistance device. In the parking assistance device according to claim 2 or 3,
The notification means includes
Processing at least one of the image data so that an image based on the latest image data and an image based on the past image data can be discriminated by visual recognition, and outputting to the display means A parking assistance device characterized by the above. In the parking assistance device according to claim 4,
The informing means processes at least one of the color tone, pattern, and brightness of the past image data so as to be different from the latest image data, and outputs the processed data to the display means. . In the parking assistance device according to any one of claims 2 to 5,
The notification means includes
A parking assistance device, characterized in that a guidance display suggesting that the past image data is displayed on the display means is output to the display means. In the parking assistance device according to any one of claims 2 to 6,
The notification means includes
A parking assistance device, characterized in that on the screen of the display means, a display for partitioning a display area of the latest image data or past image data is output. In the parking assistance device according to any one of claims 2 to 7,
The notification means includes
A parking assistance device, characterized in that a guidance sound or guidance voice suggesting that the past image data is displayed on the display means is output from a speaker.

Images