JP2010188744A - Device, method and program for supporting parking - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device, a method and a program for supporting parking, guiding a vehicle to a proper parking position with light processing burden and by a simple method. <P>SOLUTION: This device for supporting parking includes a reference image storing part 31 for storing images of a parking position in a parking lot as reference images, a correlation detecting part 33 for detecting a correlation of present images imaged by a camera installed in the vehicle to the reference images, a deciding part 34 for deciding a deviation between the present images and the reference images based on the correlation detected by the correlation detecting part 33, and a sound guide generating part 35 for generating a sound guide based on the deviation decided by the deciding part 34. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a parking support device, a parking support method, and a parking support program that support parking of a vehicle based on, for example, an image captured by an imaging unit installed in the vehicle.

  In recent years, with the generalization of automobiles and the development of information processing technology, in-vehicle electronic devices are rapidly spreading. Among them, a system for assisting driving by displaying an image captured by a camera installed in a vehicle on a display of a navigation device is generally equipped with the spread of the navigation device.

  For example, Patent Literature 1 discloses a parking assist device that appropriately notifies the driver so that the vehicle is in an optimal position during parking. This device uses a camera installed at the rear of the vehicle to capture the rear of the vehicle, and displays the captured image and the predicted travel path on the display while verifying the parking area from the position and direction of the white line in a predetermined window. The voice message is output to the driver.

JP 2000-79860 A

  By the way, in the conventional technology as described above, every time the vehicle is actually parked in the parking lot, it is necessary to perform complicated calculations such as calculation of a predicted travel locus, verification of a parking area, display of a guideline based on this, The processing burden on the apparatus increases. On the other hand, parking lots where users frequently go on a daily basis such as offices, supermarkets, convenience stores, etc. are limited, and such parking lots often do not require advanced guidance.

  The present invention solves the above-mentioned problems of the prior art, and its object is to provide a parking assistance device and a parking assistance that can be guided to an appropriate parking position by a simple method with a light processing burden. It is to provide a method and parking assistance program.

  In order to achieve the above object, the invention of claim 1 includes a reference image storage unit that stores an image of a parking position in a parking lot as a reference image, a current image captured by an imaging unit installed in a vehicle, A correlation detection unit that detects a correlation with a reference image stored in the image storage unit; a determination unit that determines a deviation between the current image and the reference image based on the correlation detected by the correlation detection unit; and the determination A voice guide generating unit that generates a voice guide based on the deviation determined by the unit.

  The parking support method of claim 5 captures the invention of claim 1 from the viewpoint of the method, and the parking support program of claim 6 captures the invention of claims 1 and 5 from the viewpoint of the computer program. It is.

  In the inventions of claims 1, 5 and 6 as described above, grasping of the deviation of the vehicle from the position to be parked for generating the voice guide is based on the correlation between the reference image stored in advance and the current image. Do. In this way, obtaining the correlation between the two images requires a lighter processing load than calculation of a predicted travel locus, verification of a parking area, display of a guideline based on this, and the like. In particular, since the user is familiar with parking lots that are frequently visited on a daily basis, even simple guidance based on the difference between the reference image and the current image is sufficiently useful.

According to a second aspect of the present invention, in the parking assist device of the first aspect, the correlation detection unit is an optical flow generation unit that generates an optical flow, and the determination unit determines a vector direction based on the optical flow. It is the vector determination part which determines, It is characterized by the above-mentioned.
In the invention of claim 2 as described above, parking assistance can be performed by a simple method of generating an optical flow and determining a vector.

According to a third aspect of the present invention, in the parking assist device according to the first or second aspect, the apparatus has a plurality of reference images according to different parking positions, and has an input unit for selecting one of the plurality of reference images. It is characterized by that.
In the invention of claim 3 as described above, the user can select a desired image suitable for the parking lot where the vehicle is currently parked from among the reference images stored in advance by using the input unit. it can.

According to a fourth aspect of the present invention, there is provided the parking support apparatus according to the first or second aspect, wherein a current position calculation unit that has a plurality of reference images and calculates a current position of the vehicle according to different parking positions; A selection unit that selects one of the plurality of reference images based on the current position calculated by the position calculation unit is provided.
In the invention of claim 4 as described above, since the parking lot to be parked is determined based on the position where the vehicle currently exists, and the reference image corresponding to the parking lot is automatically selected. This saves the user from selecting a reference image.

  According to the present invention as described above, it is possible to provide a parking support device, a parking support method, and a parking support program that are light in processing load and can be guided to an appropriate parking position by a simple method.

The functional block diagram which shows the structure of one Embodiment of this invention. The flowchart which shows the process sequence in embodiment of FIG. Explanatory drawing which shows the parking example in the best position of the vehicle in embodiment of this invention. Explanatory drawing which shows the example of a display screen corresponding to FIG. Explanatory drawing which shows the example of parking shifted to the left of the vehicle in embodiment of this invention. FIG. 6 is an explanatory diagram illustrating an example of a display screen for a current image corresponding to FIG. 5. FIG. 6 is an explanatory diagram illustrating an example of a display screen of a comparison image between a current image and a reference image corresponding to FIG. Explanatory drawing which displayed the optical flow on the comparative image of FIG. Explanatory drawing which shows the example of the vector detected by extracting the car stop in embodiment of this invention. Explanatory drawing which shows the example of a display screen at the time of selecting a reference image in other embodiment of this invention.

  Next, the best mode for carrying out the present invention (hereinafter referred to as “embodiment”) will be specifically described with reference to the drawings. In addition, although this embodiment is realizable by controlling the computer provided with the peripheral device by the program, the implementation | achievement aspect of the hardware and program in this case can be variously changed. Therefore, in the following description, virtual circuit blocks that implement the functions of the present invention and this embodiment are used. In addition to the parking assistance device and the assistance method, the present invention can also be grasped as a program as described above and a computer-readable storage medium storing such a program.

[1. Constitution〕
[1-1. overall structure〕
First, as shown in the block diagram of FIG. 1, the parking assistance device (hereinafter referred to as “this device”) in this embodiment includes a camera 1, a shutter switch 2, a control unit 3, a monitor 4, a speaker 5, and the like. Yes.

  The camera 1 is an imaging unit installed at the rear of the vehicle so that the rear of the vehicle can be imaged. A general vehicle-mounted camera can be used. The shutter switch 2 is an input unit for capturing an image captured by the camera 1 as a reference image. The shutter switch 2 may be provided in the casing of the present apparatus, may be remotely disposed via a cord, or may be incorporated in a remote controller. Note that the touch panel configured on the monitor 4 can be used as the shutter switch 2.

  The control unit 3 is a CPU that controls the entire apparatus and its peripheral circuits, and details thereof will be described later. The monitor 4 displays an image captured by the camera 1, an image stored in the control unit 3, a contrast image of these (including various interfaces such as modified and modified images, arrows, lines, operation menus, and the like). Part.

  The display content of the monitor 4 can be switched, selected, etc. by an input unit (not shown). As this input unit, a switch provided in the casing of the apparatus, a switch remotely disposed via a cord, a remote control for remote operation, a touch panel configured in the monitor 4 and the like can be considered.

  In the present embodiment, by operating the input unit, the reference image identification information (name, number, etc.) stored in the reference image storage unit 31 is displayed in a list, and when one of them is selected, the selected reference is selected. An image is displayed. The speaker 5 is means for playing back the guide voice output from the control unit 3.

[1-2. Configuration of control unit]
The control unit 3 includes a reference image storage unit 31, an image comparison unit 32, a correlation detection unit 33, a determination unit 32, a voice guide generation unit 35, a switching unit 36, and the like. The reference image storage unit 31 is a unit that stores an image captured by the camera 1 as a reference image. As the reference image, an image captured in an optimal position when the user parked the vehicle while moving backward in the parking space of the parking lot in the past is used.

  The image comparison unit 32 is a means for comparing the current image currently captured by the camera 1 with the reference image. The comparison by the image comparison unit 32 is performed by comparing the current image with the reference image and extracting common feature points. For example, it is conceivable to extract the outline of a behind car stop (curbstone etc.) or a guideline (white line etc.). There are various methods for extracting such an outline, but since it is a well-known technique, description thereof is omitted.

  The correlation detection unit 33 is means for performing correlation detection from the feature points of the current image and the reference image. This correlation detection is performed by calculating a vector corresponding to a difference (shift) between images to be compared. This can be realized by using a general optical flow generation technique. The determination unit 34 is a means for determining the direction and size of the calculated vector.

  The voice guide generator 35 is a means for outputting a predetermined voice guide according to the direction and magnitude of the vector determined by the determination unit 34. In the storage unit such as a memory, a text describing the guide contents, a table describing the correspondence between the direction and size of the vector and the text, and the like are registered, and the voice guide generating unit 35 performs voice synthesis in accordance with this. .

  The switching unit 36 is means for switching an image to be displayed on the monitor 4 to a current image captured by the camera 1, a composite image of a reference image and a current image, a composite image of an original image and a vector (arrow), and the like. The user can select which image to switch to by operating the input unit.

  The information storage unit used in the present embodiment, such as the image, text, table, interface, and the like, can be realized in any storage medium that can be used now or in the future, such as a memory or a hard disk.

[2. Action)
An example of the processing according to the present embodiment as described above will be described with reference to the flowchart of FIG. 2 and the explanatory diagrams of FIGS. 3-9, 6 is a vehicle, 7 is a vehicle stop, 8 is a guideline, 9 is a wall surface. That is, when the user moves the vehicle 6 backward according to the parking space of the parking lot, an image behind the vehicle captured by the camera 1 is input to the control unit 3 (step 201) and displayed on the monitor 4. (Step 202).

  If the reference image corresponding to the parking lot does not yet exist (step 203), the user operates the shutter switch 2 at the best position when parked (step 210). The captured image is input to the control unit 3 (step 211) and stored in the reference image storage unit 31 (step 212). When the power is turned off after the parking is completed, such as by turning off the engine of the vehicle, the display ends (step 213).

  For example, FIG. 3 is an overhead view when the vehicle moves backward in the parking space, FIG. 4A is a screen example of the monitor 4 when the vehicle is on the near side, and FIG. It is an example of a screen of the monitor 4 when the vehicle is on the rear side. If this FIG.4 (b) is the best position at the time of parking, a reference image will be preserve | saved when a user operates the shutter switch 2 at this time. At this time, identification information (for example, 001, 002, etc.) is added by default. The user may be able to attach a desired number, name, etc. by input from the input unit.

  After that, when there is an opportunity to park in the same parking lot, when the user moves backward with the vehicle 6 in accordance with the parking space, the rear image of the vehicle captured by the camera 1 is displayed in the control unit 3 as described above. (Step 201) and displayed on the monitor 4 (step 202).

  Using the input unit, the user displays a list of reference image identification information on the monitor 4 and selects a reference image of the corresponding parking lot (step 203). Then, the image comparison unit 32 compares the selected reference image with the current image being picked up and extracts feature points (step 204). Further, the switching unit 36 causes the monitor 4 to display an image obtained by comparing the reference image with the current image (step 205).

  For example, FIG. 5 is an overhead view when the vehicle is shifted to the left side in the parking space, and FIG. 6 is a screen example (current image) of the monitor 4 when the vehicle at that time is on the rear side. 7 is a screen example (comparison image) of the monitor 4 at the same position. When the vehicle is shifted to the left as shown in FIG. 5, the vehicle stop 7 and the guideline 8 in the original image of FIG. 6 are displayed shifted to the right. In this case, a comparison image between the current image at the shifted position and the reference image is as shown in FIG. 7 when the reference image is represented by a solid line and the original image is represented by a dotted line.

  Next, based on the extracted feature points, the correlation detection unit 33 generates an optical flow (step 206). Then, the determination unit 34 determines the direction and size of the vector from the generated optical flow (step 207).

  For example, when correlation detection between the reference image and the current image as shown in FIG. 7 is performed, a vector corresponding to the magnitude and direction of the shift amount can be generated. FIG. 8 is an example in which this vector is displayed on the screen, and the target parking position can be known by determining the direction and magnitude of this vector.

  Then, according to the determined direction and magnitude of the vector, the voice guide generator 35 generates a voice guide (step 208), which is reproduced by the speaker 5 (step 209). For example, FIG. 9 shows an example of a vector detected by extracting the car stop 7 and shows an example of what kind of voice guidance is performed for this.

  That is, as shown in FIG. 9A, when the vector is directly above, it means that the direction of the vehicle is straight toward the desired stop position. In this case, for example, it may be possible to output a voice guide saying “If it goes straight down, it is the best parking position”. As shown in FIG. 9 (b), when the vehicle is deviating to the left when viewed from directly above and is about to be parked, for example, a voice guide saying “It is shifted to the right side from the parking position” is output. It is possible.

  Also, as shown in FIG. 9C, when the vehicle position is close to the normal parking position in the depth direction, for example, a voice guide saying “the best parking position is in several centimeters” is output. It is possible. Furthermore, as shown in FIG. 9D, when the vehicle is stopped slightly shifted to the left side with respect to the desired position, for example, “It is shifted to the right by several centimeters from the parking position. It is possible to output a voice guide.

  If the vehicle is parked exactly at the regular parking position, the vector is zero. In this case, for example, it may be possible to output a voice guide that “the vehicle has been parked at the best position”. After parking, the display is terminated when the power is turned off by turning off the engine of the vehicle (step 212).

[3. effect〕
According to the present embodiment as described above, the deviation of the vehicle 6 from the position to be parked is grasped based on the correlation between the reference image stored in advance and the current image, and an appropriate voice guide is generated. Thus, parking can be supported. In particular, in the present embodiment, a correlation between two images is obtained by generating an optical flow, and a shift is grasped by vector determination. Therefore, each time the vehicle actually goes to the parking lot, it is possible to greatly reduce the processing load, rather than calculating the predicted travel locus, verifying the parking area, displaying the guideline based on this, and the like.

  In this way, even by providing guidance based on the deviation from the reference image, the user is well-informed about parking lots that are frequently visited on a daily basis. In addition, the user can easily obtain a desired reference image suitable for a parking lot where the vehicle is currently parked by selecting the reference image displayed as a list on the monitor 4.

[4. Other embodiments]
In addition, this invention is not limited to said embodiment. Each part of the above embodiment can also be used as a navigation device. For example, the storage medium constituting the reference image storage unit 31 and other storage units can also be used as a memory of a navigation device, a hard disk, or the like. The shutter switch 2 and other input units may be configured by a switch or touch sensor provided on the front panel of the navigation device, a remote control unit that can be remotely operated, or the like. Note that a voice recognition unit may be provided so that operations similar to those performed by the shutter switch 2 and other input units may be performed in response to voice input from a microphone.

  The monitor 4 may be used as the display of the navigation device or may be configured as a separate body. The touch panel of the display of the navigation device can also be configured as the shutter switch 2 and other input units. The speaker 5 may also be used as a navigation device.

  An interface for selecting a reference image is also free. For example, as shown in FIG. 10, the user may be able to select from a reference image displaying a plurality of icons using an input unit.

  Regarding the selection of the reference image, the following modes can be considered. That is, when storing a reference image in a parking lot, a current position is detected by a current position detection unit such as a locator or a separately provided locator in the navigation device, and this position information is associated with the reference image and a reference image storage unit 31 is stored.

  Next, when the selection unit set in the control unit 3 is parked in the parking lot, the reference image has position information that matches or is within a predetermined distance with respect to the current position obtained from the current position detection unit. Is selected from the reference image storage unit 31 as a comparison target by the image comparison unit 32. As a result, a reference image suitable for comparison with the current image is automatically selected, so that the user can save time and effort.

  The locator is a means for calculating the current position (vehicle position) of the vehicle by GPS positioning based on the signal received by the GPS tuner. By taking map matching with data, accuracy can be ensured, but since it is a well-known technique, description thereof is omitted.

  Further, for example, it is conceivable to provide a distance detection unit such as a sensor capable of detecting a distance to an external object such as the car stopper 7 or the wall surface 9. In this case, when the vehicle is parked in the parking space, the reference image and the distance are associated with each other for each predetermined distance and stored in the reference image storage unit 31 as the distance to the object of the vehicle changes.

  Next, when the selection unit set in the control unit 3 parks in the parking lot, a reference image that matches or is within a predetermined distance with respect to the distance obtained from the distance detection unit is displayed as an image comparison unit 32. Is selected from the reference image storage unit 31 as a comparison target. As a result, a reference image suitable for comparison with the current image is automatically selected according to the positional relationship between the vehicle and the parking space, so that more accurate guidance is possible.

  In addition, as a distance detection part, what can detect the distance with an external target object should just be used. In general, a sensor that calculates a distance based on a time until a transmitted optical signal is reflected by an object and received is considered, but the present invention is not limited to this. The distance may be determined from the ratio of the size of the same object in the reference image and the current image.

  Any known technique can be applied to the correlation detection method. It is possible to simplify the processing by determining only the direction or size of the vector and instructing only the correction direction of the shift or only the correction distance of the shift with the voice guide. The object for recognizing the parking space is not limited to those exemplified in the above embodiment.

  The installation position of the imaging unit may be any position on the vehicle. For example, the horizontal direction may be imaged for parallel parking. Any known technique can be applied to the method for generating and outputting the voice guide. For example, any of audio data stored in a storage unit such as a memory in advance may be selected and output.

DESCRIPTION OF SYMBOLS 1 ... Camera 2 ... Shutter switch 3 ... Control part 4 ... Monitor 5 ... Speaker 6 ... Vehicle 7 ... Car stop 9 ... Wall 31 ... Reference image memory | storage part 32 ... Image comparison part 33 ... Correlation detection part 34 ... Determination part 35 ... Sound guide Generating unit 36 ... switching unit

Claims (6)

A reference image storage unit for storing an image of a parking position in the parking lot as a reference image;
A correlation detection unit for detecting a correlation between a current image captured by an imaging unit installed in a vehicle and a reference image stored in the image storage unit;
A determination unit for determining a deviation between the current image and the reference image based on the correlation detected by the correlation detection unit;
A voice guide generating unit that generates a voice guide based on the deviation determined by the determination unit;
A parking assistance device comprising: The correlation detection unit is an optical flow generation unit that generates an optical flow,
The parking assist device according to claim 1, wherein the determination unit is a vector determination unit that determines a vector direction based on the optical flow. According to different parking positions, has multiple reference images,
The parking support apparatus according to claim 1, further comprising an input unit that selects any one of the plurality of reference images. According to different parking positions, has multiple reference images,
A current position calculation unit for calculating the current position of the vehicle;
The parking assist device according to claim 1, further comprising a selection unit that selects one of the plurality of reference images based on the current position calculated by the current position calculation unit. The computer or electronic circuit has a reference image storage unit, a correlation detection unit, a determination unit, a voice guide generation unit,
A process in which the reference image storage unit stores an image of a parking position in a parking lot as a reference image;
A process for detecting a correlation between the current image captured by the imaging unit installed in the vehicle and the reference image stored in the image storage unit, the correlation detection unit;
A process in which the determination unit determines a shift between a current image and a reference image based on the correlation detected by the correlation detection unit;
The voice guide generating unit generates a voice guide based on the deviation determined by the determination unit;
The parking assistance method characterized by performing. A program executable by a computer,
In the computer,
A process of storing an image of a parking position in a parking lot as a reference image;
A process for detecting a correlation between a current image captured by an imaging unit installed in a vehicle and a reference image;
A process of determining a shift between the current image and the reference image based on the detected correlation;
A process of generating a voice guide based on the determined deviation;
A parking assistance program characterized in that

Images