JP2012136206A - System and method for parking control - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technology for parking control capable of appropriately parking a vehicle within a parking frame on the basis of the parking frame of a big recognition probability when the parking frame is recognized on the basis of a car outside image obtained from a plurality of cameras.SOLUTION: The system for parking control recognizes a parking frame on the basis of a car outside image obtained by a plurality of cameras. Then, the parking control system executes vehicle parking control on the basis of, among a plurality of recognized parking frames, a parking frame of a biggest recognition probability. Then, vehicles can be appropriately parked in the parking frame.

Description

  The present invention relates to a technology of a parking control system that performs parking control for controlling the behavior of a vehicle and parks the vehicle in a parking frame.

  In recent years, a parking control system for parking a vehicle in a parking frame has been developed.

  More specifically, the parking control system derives a route for recognizing the position of the parking frame based on the outside image captured by the camera provided in the vehicle and guiding the current frame to the recognized parking frame position. Then, the parking control system controls the vehicle to follow the derived route and parks it within the parking frame.

  Thereby, the user can easily park the vehicle in the parking frame. Such a technique is described in Patent Document 1, for example.

JP 2010-195224 A

  However, when the accuracy of recognizing the parking frame based on the outside image is small, there is a problem that the vehicle cannot be parked appropriately in the parking frame.

  This invention is made | formed in view of the said subject, and it aims at providing the technique of the parking control which parks a vehicle appropriately in a parking frame based on the parking frame with the large recognition accuracy.

  In order to solve the above-mentioned problem, the invention of claim 1 is a parking control system that is mounted on a vehicle, performs parking control for controlling the behavior of the vehicle, and parks the vehicle in a parking frame. Recognizing means for acquiring a plurality of outside images obtained by a plurality of mounted cameras, a recognition means for recognizing the image of the parking frame included in the plurality of outside images, and the plurality of outside images. A selection unit that selects the image of the parking frame having the highest recognition accuracy among the images of the parking frame, and a control unit that executes the parking control based on the selected image of the parking frame. It is characterized by.

  According to a second aspect of the present invention, in the parking control system according to the first aspect, the recognizing unit detects an edge of the parking frame image of the outside image, and the selecting unit is based on the edge. The accuracy of the recognition is obtained.

  Further, the invention according to claim 3 is the parking control system according to claim 1 or 2, wherein the plurality of cameras include a side camera that captures a side of the vehicle, and the selection unit is configured to perform the parking control. When the vehicle is moving backward in a curve, the parking frame image included in the outside image acquired by the side camera is selected.

  Further, the invention of claim 4 is the parking control system according to any one of claims 1 to 3, wherein the plurality of cameras include a rear camera that captures the rear of the vehicle, and the selection means includes the parking control. When the vehicle starts to reverse, the image of the parking frame included in the outside image acquired by the rear camera is selected.

  According to a fifth aspect of the present invention, in the parking control system according to any one of the first to fourth aspects, based on the recognized image of the parking frame before starting the reverse of the vehicle by the parking control. First deriving means for deriving the reverse start position of the vehicle and the position of the parking frame on the virtual map, and the vehicle within the parking frame based on the derived reverse start position and the position of the parking frame Second derivation means for deriving a route on the virtual map to be guided to the vehicle, and starting the backward movement of the vehicle by the parking control, and then on the virtual map based on the recognized image of the parking frame Determining means for determining the current position of the vehicle, wherein the control means controls the behavior of the vehicle so that the current position matches the route.

  According to a sixth aspect of the present invention, in the parking control system according to any one of the first to fifth aspects, the lenses of the plurality of cameras are fish-eye lenses.

  According to a seventh aspect of the present invention, in the parking control system according to any one of the first to sixth aspects, the generation means for generating a composite image in which the periphery of the vehicle is viewed from a virtual viewpoint based on the plurality of outside images. And a display means for displaying the composite image during execution of the parking control.

  The invention according to claim 8 is a parking control method for performing parking control for controlling the behavior of the vehicle to park the vehicle in a parking frame, and (a) obtained by a plurality of cameras mounted on the vehicle. A step of acquiring a plurality of images outside the vehicle, (b) a step of recognizing an image of the parking frame included in the plurality of images outside the vehicle, and (c) the parking frame recognized by each of the plurality of images outside the vehicle. A step of selecting the image of the parking frame having the highest recognition accuracy among the images of (d), and (d) a step of executing the parking control based on the selected image of the parking frame. To do.

  The invention according to claim 9 is an image processing device for processing an image, the acquisition means acquiring a plurality of outside images obtained by a plurality of cameras mounted on the vehicle, and the plurality of outside images. A recognizing unit for recognizing the image of the parking frame, a selecting unit for selecting the image having the highest recognition accuracy among the images of the parking frame recognized by the plurality of images outside the vehicle, and the selected parking An image processing apparatus comprising: an output unit that outputs information based on a frame image to a parking control device that controls the behavior of the vehicle and parks the vehicle in a parking frame.

  According to the first to ninth aspects of the present invention, since the parking control of the vehicle is executed based on the parking frame having the highest recognition accuracy among the plurality of parking frames, the vehicle can be appropriately parked in the parking frame. .

  According to a second aspect of the present invention, a parking frame suitable for parking control is selected from a plurality of parking frames in order to obtain the accuracy of recognizing the parking frame image by the edge of the parking frame image in the outside image. can do.

  According to the invention of claim 3, when the vehicle is moving backward in a curve by the parking control, the image of the parking frame included in the outside image acquired by the side camera is selected. Parking control can be executed based on the image of the parking frame with the best recognition accuracy among the images.

  According to the fourth aspect of the present invention, when starting the reverse movement of the vehicle by the parking control, the image of the parking frame included in the image outside the vehicle captured by the rear camera is selected. The parking control can be executed based on the image of the parking frame with the best accuracy.

  Further, according to the invention of claim 5, the vehicle current position is determined so that the current position determined based on the image of the parking frame matches the route derived based on the image of the parking frame. Can be restored even if the vehicle goes off the route during parking control.

  According to the sixth aspect of the present invention, the outside of the vehicle in a relatively wide range can be photographed by using a fisheye lens as the lens of the plurality of cameras. As a result, the image of the parking frame can be firmly captured in the outside image.

  In addition, according to the invention of claim 7, the images outside the vehicle photographed by a plurality of cameras are used for generating a composite image in which the vehicle is viewed from a virtual viewpoint, and are also used for the function of parking control. Effectively use images outside the car taken by the camera.

FIG. 1 is a diagram illustrating a parking control system block. FIG. 2 is a diagram illustrating the vehicle. FIG. 3 is a diagram illustrating the vehicle. FIG. 4 is a diagram illustrating a vehicle and an outside image. FIG. 5 is a diagram for explaining the process of synthesizing the outside image. FIG. 6 is a diagram illustrating garage parking control. FIG. 7 is a diagram illustrating an image outside the vehicle. FIG. 8 is a diagram illustrating garage parking control. FIG. 9 is a diagram illustrating an image outside the vehicle. FIG. 10 is a diagram illustrating an image outside the vehicle. FIG. 11 is a diagram illustrating an image outside the vehicle. FIG. 12 is a diagram illustrating an image outside the vehicle. FIG. 13 is a diagram showing a control flow. FIG. 14 is a diagram showing a control flow. FIG. 15 is a diagram showing a control flow. FIG. 16 is a diagram illustrating an outside-vehicle image.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
FIG. 1 is a diagram illustrating a configuration of the parking control system SY according to the first embodiment. The parking control system SY is mounted on the vehicle 1 and performs parking control for controlling the behavior of the vehicle 1 to park the vehicle 1 in the parking frame.

  More specifically, the parking control system SY acquires a plurality of outside images obtained by a plurality of cameras mounted on the vehicle 1. And parking control system SY recognizes the image of the parking frame contained in a plurality of outside pictures. And the parking control system SY selects the image of the parking frame with the highest recognition accuracy among the images of the parking frame recognized in each of the plurality of images outside the vehicle. And the parking control system SY performs parking control based on the image of the selected parking frame.

(Configuration of parking control system)
In order to explain the function exhibited by the parking control system SY, first, the configuration of the parking control system SY will be described with reference to FIG.

  The parking control system SY includes an image processing device 2, a display 12, a parking control device 13, and the like.

  The image processing device 2 includes a front camera 3, a left side camera 4, a right side camera 5, a rear camera 6 (hereinafter referred to as cameras 3 to 6), a vehicle speed sensor 7, a steering angle sensor 8, a parking control device 13, and a display. 12 etc. are electrically connected by a cable.

  The parking control device 13 is electrically connected to the image processing device 2, the engine control device 9, the steering device 10, the brake device 11, and the like through cables.

  As shown in FIG. 2, the front camera 3 is provided in the vicinity of the license plate mounting position at the front end of the vehicle 1, and its optical axis is directed in the straight traveling direction of the vehicle 1.

  The right side camera 5 is provided on the right side mirror as shown in FIG. 2, and its optical axis is directed to the outside along the right direction based on the straight traveling direction of the vehicle 1.

  The left side camera 4 is provided on the left side mirror as shown in FIG. 2, and its optical axis is directed to the outside along the left direction based on the straight traveling direction of the vehicle 1.

  As shown in FIG. 2, the rear camera 6 is provided in the vicinity of the license plate mounting position at the rear end of the vehicle 1, and its optical axis is directed in the direction opposite to the straight traveling direction of the vehicle 1.

  Note that the attachment position of the front camera 3 and the rear camera 6 is preferably approximately the center in the left-right direction, but may be a position slightly shifted in the left-right direction from the center in the left-right direction.

  Fisheye lenses are employed as the lenses of the cameras 3 to 6, and the cameras 3 to 6 have an angle of view of 180 degrees or more. For this reason, by using these four image capturing units, it is possible to capture the areas H1 to H4 covering the entire periphery of the vehicle 1 as shown in FIG.

  The vehicle speed sensor 7 outputs the speed at which the wheel rotates when the vehicle 1 moves to the image processing device 2 as a vehicle speed signal. The vehicle speed sensor 7 is installed in the engine room of the vehicle 1.

  The steering angle sensor 8 outputs to the image processing device 2 the angle when the steering wheel provided in the vehicle 1 is steered by the user as a steering angle signal. The steering angle sensor 8 is installed in the vicinity of the rotating shaft of the steering.

  The image processing apparatus 2 includes an acquisition unit 20, an image processing unit 21, a control unit 22, and an output unit 23.

  The acquisition unit 20 is, for example, an interface, and acquires an image outside the vehicle photographed by the cameras 3 to 6, a vehicle speed signal output by the vehicle speed sensor 7, a steering angle signal output by the steering angle sensor 8, and the like.

  The image processing unit 21 includes a recognition unit 30, a selection unit 31, a generation unit 32, a display unit 33, and the like. The image processing unit 21 is, for example, a hardware circuit such as an ASIC (Application Specific Integrated Circuit).

  The recognizing unit 30 executes processing for recognizing parking frame images included in a plurality of outside images taken by the cameras 3 to 6. Details of this recognition processing executed by the recognition unit 30 will be described later.

  The selection unit 31 executes a process of selecting the image of the parking frame having the highest recognition accuracy among the images of the parking frame recognized by the plurality of images outside the vehicle. Details of the selection process executed by the selection unit 31 will be described later.

  The generation unit 32 generates a composite image showing the vehicle 1 and the surroundings of the vehicle viewed from the virtual viewpoint, based on a plurality of outside images obtained by the cameras 3 to 6. Details of the generation processing executed by the generation unit 32 will be described later.

  The display unit 33 executes a process of causing the display 12 to display a composite image generated during the execution of parking control. Details of the display processing executed by the display unit 33 will be described later.

  The control unit 22 is a microcomputer including a CPU, a ROM, a RAM, and the like, for example. The CPU realizes various functions by performing arithmetic processing according to various programs stored in the ROM. The control unit 22 includes a derivation unit 34 and a determination unit 35 that realize some of the various functions.

  The deriving unit 34 derives the reverse start position of the vehicle 1 and the position of the parking frame on the virtual map based on the recognized image of the parking frame before starting the reverse of the vehicle 1 by the parking control. Further, the deriving unit 34 derives a route on the virtual map for guiding the vehicle 1 into the parking frame based on the derived backward start position and the position of the parking frame. Details of the derivation process executed by the derivation unit 34 will be described later.

  The determination unit 35 determines the current position of the vehicle 1 on the virtual map based on the recognized image of the parking frame after starting the backward movement of the vehicle 1 by the parking control. Details of this determination processing executed by the determination unit 35 will be described later.

  The output unit 23 is, for example, an interface, and outputs the composite image generated by the generation unit 32 included in the image processing unit 21 to the display 12. Further, the output unit 23 outputs information based on the image of the parking frame selected by the selection unit 31 included in the image processing unit 31 to the parking control device 13.

  The parking control device 13 is, for example, a box containing an electronic board on which electronic components such as a microcomputer are mounted. The parking control device 13 includes a parking control unit 36 and the like.

  The parking control unit 36 controls the behavior of the vehicle 1 such as acceleration / deceleration, direction change, and stop of the vehicle 1 in cooperation with the engine control device 9, the steering device 10, the brake device 11, and the like. The vehicle 1 is parked at the position.

  That is, the parking control unit 36 acquires information based on the image of the selected parking frame from the image processing device 2. And the parking control part 36 produces | generates the virtual map utilized when parking control by the information based on the image of the acquired parking frame. And the parking control part 36 performs the parking control which makes the vehicle 1 drive | work and park in a parking frame so that the present position of the vehicle 1 on the produced | generated virtual map may correspond with the route | root on a virtual map.

<Generation process>
Next, the generation process executed by the generation unit 32 will be described in detail.

  The acquisition unit 20 included in the image processing apparatus 2 acquires from the cameras 3 to 6 the vehicle outside images Z1 to Z4 as shown in FIG. The acquisition unit 20 outputs the acquired outside-vehicle images Z1 to Z4 to the image processing unit 21.

  The image processing unit 21 stores the input outside-vehicle images Z1 to Z4 in a volatile storage unit (not shown).

  The generation unit 32 included in the image processing unit 21 projects the outside-vehicle images Z1 to Z4 stored in a volatile storage unit (not shown) onto a three-dimensional curved surface P in a virtual three-dimensional space as shown in FIG. The three-dimensional curved surface P has, for example, a substantially hemispherical shape (a bowl shape), and a central portion (a bottom portion of the bowl) is determined as a position where the vehicle 1 exists. A correspondence relationship is determined in advance between the position of each pixel included in the outside-vehicle images Z1 to Z4 and the position of each pixel of the solid curved surface P. For this reason, the value of each pixel of the solid curved surface P is determined based on this correspondence and the value of each pixel included in the outside-vehicle images Z1 to Z4.

  The correspondence between the positions of the pixels of the outside images Z1 to Z4 and the positions of the pixels of the three-dimensional curved surface P is the arrangement of the four cameras in the vehicle 1, that is, the arrangement of the cameras 3 to 6 (the distance between each other, the ground height, the light Depending on the shaft angle. For this reason, the table data indicating this correspondence relationship is included in the vehicle type data stored in the nonvolatile storage unit (not shown).

  Further, polygon data indicating the shape and size of the vehicle body included in the vehicle type data is used, and a vehicle image which is a polygon model indicating the three-dimensional shape of the vehicle 1 is virtually configured. The configured vehicle image is arranged in a substantially hemispherical central portion defined as the position of the vehicle 1 in the three-dimensional space in which the three-dimensional curved surface P is set.

  Further, the virtual viewpoint AGL is set by the generation unit 32 for the three-dimensional space in which the three-dimensional curved surface P exists. The virtual viewpoint AGL is defined by the viewpoint position and the viewing direction, and is set at an arbitrary viewpoint position corresponding to the periphery of the vehicle 1 in this three-dimensional space toward an arbitrary viewing direction.

  Then, according to the set virtual viewpoint AGL, a necessary area on the three-dimensional curved surface P is cut out as an image. The relationship between the virtual viewpoint AGL and a necessary area on the three-dimensional curved surface P is determined in advance, and stored in advance in a non-volatile storage unit (not shown) as table data. On the other hand, rendering is performed with respect to a polygon model in accordance with the set virtual viewpoint AGL, and the resulting image of the two-dimensional vehicle 1 is superimposed on the clipped image. As a result, a composite image showing a state in which the vehicle 1 and the periphery of the vehicle 1 are viewed from an arbitrary virtual viewpoint AGL is generated.

  For example, when the virtual viewpoint AGL1 in which the viewpoint position is just above the center of the position of the vehicle 1 and the visual field direction is directly below is set, the vehicle 1 and the vehicle 1 so that the vehicle 1 is looked down from almost directly above the vehicle 1. A composite image G1 is generated that indicates the area around the. Further, as shown in the figure, when the virtual viewpoint AGL2 in which the viewpoint position is the left rear of the position of the vehicle 1 and the visual field direction is a substantially forward direction in the vehicle 1 is set, the entire periphery from the left rear of the vehicle 1 is set. A composite image G2 indicating the vehicle 1 and the area around the vehicle 1 is generated so as to overlook the vehicle.

  In the case of actually generating a composite image, it is not necessary to determine the values of all the pixels of the three-dimensional curved surface P, and only the values of the pixels in the area necessary corresponding to the set virtual viewpoint AGL are outside the vehicle. By determining based on the images Z1 to Z4, the processing speed can be improved.

  In the image processing device 2, by using such a function of the generation unit 32, a composite image viewed from an arbitrary viewpoint around the vehicle 1 can be generated and the generated composite image can be displayed on the display 12. .

<Display processing>
Next, display processing executed by the display unit 33 will be described in detail.

  The display unit 33 executes a process of causing the display 12 to display a composite image generated during the execution of parking control. For this reason, the user can see how the vehicle 1 is parked in the parking area on the display 12.

<Recognition process>
Next, the recognition process executed by the recognition unit 30 will be described in detail. The recognition part 30 recognizes the image of the parking frame contained in the several image outside a vehicle obtained for every predetermined period (for example, 100 ms) with the cameras 3-6.

  Note that the image of the parking frame includes at least side lines located near both sides of the vehicle 1 when the vehicle 1 is parked. The side lines L1 and L2 are, for example, long thin U-shaped white lines as shown in FIG.

  Recognition of the image of the parking frame included in the plurality of images outside the vehicle by the recognition unit 30 is executed by detecting the edge of the image of the parking frame in the image outside the vehicle. That is, the recognition unit 30 performs edge detection processing on the monochrome image based on the acquired outside-vehicle image. The recognizing unit 30 recognizes the image of the parking frame based on the length and thickness of the image of the object grasped by the detected edge.

<Parking method>
Here, a parking control parking method executed by the parking control system SY will be described. The parking method executed by the parking control system SY is so-called “garage parking”. The “garage parking” executed by the parking control system SY is executed according to the following procedure.

  First, in the parking lot as shown in FIG. 6, when the vehicle 1 is in the initial position P1 and receives an instruction to start parking, the parking control system SY moves the vehicle 1 from the initial position P1 to the middle position. Pass P2 and advance to the reverse start position P3. Then, the parking control system SY stops the vehicle 1 at the reverse start position P3. The parking control system SY moves the vehicle 1 backward from the reverse start position P3 and stops at the parking position PP in the parking frame.

  Hereinafter, this “garage parking” control is referred to as “garage parking control”.

<Derivation process>
First, details of the derivation process executed by the derivation unit 34 will be described. The derivation process includes a basic derivation process, a first derivation process, and a second derivation process.

<Basic derivation process>
First, details of the basic derivation process executed by the derivation unit 34 will be described. When the “garage parking control” is executed and the vehicle 1 moves from the initial position P1 to the reverse start position P3 shown in FIG. 6, the recognition unit 30 recognizes images of a plurality of parking frames. Based on the recognized plurality of parking frames, the deriving unit 34 derives the angular positions ED1 to ED6 on the vehicle entry side of the parking frame image of the side lines L1 to L5 included in the plurality of parking frames.

  The angular positions ED1 to ED6 are positions that intersect when the vertical endmost edge and the horizontal endmost edge of the side lines L1 to L5 are respectively extended.

<First derivation process>
Next, details of the first derivation process executed by the derivation unit 34 will be described. The parking position PP is derived based on the image of the parking frame included in the outside image as shown in FIG. 7 taken by the rear camera 6 when the vehicle 1 is temporarily stopped at the reverse start position P3.

  That is, the deriving unit 34 derives the parking position PP between the angular position ED1 and the angular position ED2 that appear in the predetermined area D1 of the outside image taken by the rear camera 6 when the vehicle 1 is stopped at the reverse start position P3. To do.

  Next, the reverse start position P3 of the vehicle 1 on the virtual map as shown in FIG. 8 based on the image of the parking frame recognized before starting the control for moving the vehicle 1 backward by the “garage parking control”, The deriving unit 34 derives the angular positions ED1 and ED2. The angular positions ED1 and ED2 are relative positions on the virtual map with respect to the reverse start position P3. In this case, the reference point BP of the vehicle 1 is located at the reverse start position P3 on the virtual map. The reference point BP is a point that is relatively fixed to the vehicle 1 and indicates the position of the vehicle 1 on the virtual map.

  This virtual map is defined by XY coordinates when the vehicle 1 is looked down from a virtual viewpoint. In this virtual map, the reverse start position P3 of the vehicle 1 is set to the origin of the XY coordinates (X = 0, Y = 0). Therefore, the reference point BP of the vehicle 1 on the virtual map is located at the origin of the XY coordinates. Further, the derived angular positions ED1 and ED2 are set to appropriate positions in the XY coordinates.

<Second derivation process>
Next, details of the second derivation process performed by the derivation unit 34 will be described. Based on the derived reverse start position P3 and the parking position PP, the deriving unit 34 derives a route R on the virtual map that guides the vehicle 1 into the parking frame.

  The route R on the virtual map is derived in consideration so that the vehicle 1 can be guided from the reverse start position P3 to the parking position PP in the shortest time. Further, the route R on the virtual map prevents the vehicle 1 from contacting an object (for example, another parked vehicle) that may exist outside the side lines L1 and L2 of the parking frame when the vehicle 1 is guided. Is taken into account.

<Selection process>
Next, the selection process executed by the selection unit 31 will be described in detail. The selection unit 31 selects the image of the parking frame with the highest recognition accuracy among the images of the parking frame recognized by the recognition unit 30 based on each of the plurality of outside images obtained by the plurality of cameras mounted on the vehicle. .

  The accuracy to be recognized is, for example, the degree of certainty when the length or thickness of the image of the object grasped by the edge detected by the recognition unit 30 is the length or thickness of the image of the parking frame. is there. The certainty is relatively large, for example, when the edge length is relatively long without interruption in the region below the center of the vehicle outside image or when the edge thickness is relatively thick. Note that the accuracy is considered to be the accuracy of detecting the image of the parking frame.

  For example, when the reference point BP of the vehicle 1 is located at the midway position P4 shown in FIG. 8, the recognition accuracy of the image of the parking frame recognized based on the outside image captured by the right side camera 5 shown in FIG. However, when the accuracy of recognizing the image of the parking frame recognized based on the outside image captured by the left side camera 4 and the rear camera 6 is larger than the accuracy of recognizing the parking frame image based on the right side camera 5, the selection unit 31 selects the parking frame image. Is done.

  Further, for example, when the reference point BP of the vehicle 1 is located at the midway position P5 shown in FIG. 8, the recognition of the parking frame image recognized based on the vehicle outside image taken by the right side camera 5 shown in FIG. When the accuracy of the parking frame is greater than the accuracy of recognizing the image of the parking frame recognized based on the outside image captured by the left side camera 4 and the rear camera 6, the image of the parking frame based on the right side camera 5 is selected by the selection unit 31. Selected by.

  Further, for example, when the reference point BP of the vehicle 1 is located at the midway position P6 shown in FIG. 8, the accuracy of recognizing the image of the parking frame recognized based on the outside image taken by the rear camera 6 shown in FIG. When the accuracy of recognizing the image of the parking frame recognized based on the outside image taken by the left side camera 4 and the right side camera 5 is larger than the accuracy of recognizing the parking frame based on the rear camera 6, the selection unit 31 selects the parking frame image. The

  Further, for example, when the reference point BP of the vehicle 1 is located at the midway position P7 shown in FIG. 8, the accuracy of recognizing the image of the parking frame recognized based on the outside image taken by the rear camera 6 shown in FIG. When the accuracy of recognizing the image of the parking frame recognized based on the outside image taken by the left side camera 4 and the right side camera 5 is larger than the accuracy of recognizing the parking frame based on the rear camera 6, the selection unit 31 selects the image of the parking frame. The

<Judgment process>
Next, the determination process executed by the determination unit 35 will be described in detail.

  The determination unit 35 starts the control for moving the vehicle 1 backward by the “garage parking control”, and then based on the recognized image of the parking frame, the current reference point BP of the vehicle 1 on the virtual map, that is, the vehicle 1. The current position of is determined.

  Specifically, first, during the movement of the vehicle 1 from the reverse start position P3 to the parking position PP, the angular position ED1 of the image of the parking frame selected by the selection unit 31 at a predetermined period (for example, 120 ms), The determination unit 35 determines a relative position on the virtual map between the ED 2 and the reference point BP of the vehicle 1.

  Then, the relative position (hereinafter referred to as the initial relative position) on the virtual map of the reverse start position P3 of the vehicle 1 and the angular positions ED1 and ED2 already known and the relative position (hereinafter referred to as the current position). And the position of the reference point BP of the vehicle 1 on the virtual map is determined. That is, the determination unit 35 determines the current position of the vehicle 1 on the virtual map at a predetermined cycle from when the vehicle 1 starts the control for moving the vehicle 1 backward by “garage parking control”.

  For example, when the reference point BP of the vehicle 1 is located at an intermediate position P4 shown in FIG. 8, the image of the parking frame recognized based on the outside image shown in FIG. When selected, the determination unit 35 determines the current relative position on the virtual map between the angular position ED1 of the image of the parking frame and the reference point BP assumed from the image of the vehicle 1 shown in the image outside the vehicle. Then, the determination unit 35 determines the current position of the vehicle 1 based on the initial relative position and the current relative position.

  Further, for example, when the reference point BP of the vehicle 1 is located at the midway position P5 shown in FIG. 8, the image of the parking frame recognized based on the outside image shown in FIG. 10 taken by the right side camera 5 is selected. When selected by 31, the determination unit 35 determines the relative position on the virtual map between the angular position ED1 of the image of the parking frame and the reference point BP assumed from the image of the vehicle 1 shown in the outside image. . Then, the determination unit 35 determines the current position of the vehicle 1 based on the initial relative position and the current relative position.

  Further, for example, when the reference point BP of the vehicle 1 is located at an intermediate position P6 shown in FIG. 8, the image of the parking frame recognized based on the outside image shown in FIG. When selected, the determination unit 35 determines a relative position on the virtual map between the angular position ED2 of the image of the parking frame and the reference point BP assumed from the image of the vehicle 1 shown in the image outside the vehicle. Then, the determination unit 35 determines the current position of the vehicle 1 based on the initial relative position and the current relative position.

  Further, for example, when the reference point BP of the vehicle 1 is located at an intermediate position P7 shown in FIG. 8, the image of the parking frame recognized based on the outside image shown in FIG. When selected, the relative position on the virtual map between the angular position ED1X facing the angular position ED1 of the image of the parking frame and the reference point BP assumed from the image of the vehicle 1 reflected in the image outside the vehicle 35 determines. Then, the determination unit 35 determines the current position of the vehicle 1 based on the initial relative position and the current relative position.

  In this case, the determination is made based on the angular positions ED1X and ED2X of the side lines L1 and L2 included in the parking frame, which are derived by the deriving unit 34, on the opposite side to the vehicle entry side of the parking frame image. The reason is that the accuracy of recognizing the angular position based on the image outside the vehicle photographed by the rear camera is higher than that recognized based on the image outside the vehicle photographed by another camera.

  In addition, since the image outside a vehicle image | photographed with the cameras 3-6 provided with a fisheye lens is distorted, the present relative position of the vehicle 1 is determined in consideration of the distortion.

<Parking control process>
Next, the parking control process executed by the parking control unit 36 will be described in detail. The parking control process includes a first parking control process and a second parking control process. This parking control process is the aforementioned “garage parking control”.

<First parking control process>
First, details of the first parking control process will be described.

  When receiving an instruction to start “garage parking control” from the user, the parking control unit 36 travels in parallel with a plurality of parking frames located in parallel from the initial position P1 shown in FIG. To control.

  When the vehicle 1 is traveling, when receiving an operation for selecting a parking frame to be parked from the user, the parking control unit 36 moves the rear part of the vehicle 1 to the vehicle entrance side of the selected parking frame. The vehicle 1 is stopped. At that time, the parking control unit 36 stops the vehicle 1 such that the longitudinal direction of the vehicle 1 and the long side direction of the parking frame are at a predetermined angle (for example, 25 degrees). That is, the parking control unit 36 stops the vehicle 1 at the reverse start position P3 in the positional relationship as shown in FIG.

<Second parking control process>
Next, the second parking control process will be described.

  The parking control unit 36 performs parking control that causes the vehicle 1 to travel and park in the parking frame so that the current position of the vehicle 1 on the virtual map matches the route R on the virtual map.

  The control amount of the vehicle 1 for the current position of the vehicle 1 on the virtual map to follow the route R on the virtual map is determined at the reverse start position P3, and “garage parking control” is executed with the control amount, that is, Assume that open loop control is executed. In this case, if the tire provided on the vehicle 1 slips when the vehicle 1 travels, the current position of the vehicle 1 may deviate from the route R on the virtual map, and the vehicle 1 may not be properly parked at the parking position PP. There is.

  Therefore, the parking control is executed to drive the vehicle 1 and park in the parking frame so that the current position of the vehicle 1 on the virtual map acquired in real time matches the route R on the virtual map, that is, the feedback control is executed. To do. For this reason, even if the current position of the vehicle 1 deviates from the route R on the virtual map, it can be returned to the route R. As a result, the vehicle 1 can be appropriately parked at the parking position PP.

<Control flow>
Next, a control flow executed by the parking control system SY will be described.

The parking control system SY starts the first control flow shown in FIG. 13 for a predetermined period (hereinafter referred to as the first control flow) when the system of the vehicle 1 is activated by the user and the button for executing “garage parking control” is operated. 1 cycle). Further, in that case, the parking control system SY executes the second control flow shown in FIG. 14 in a cycle longer than the first cycle (hereinafter referred to as the second cycle). Moreover, the parking control system SY performs the 3rd control flow shown in FIG. 15 in a period (henceforth a 3rd period) longer than a 2nd period in that case. The second period is, for example, 33 ms, and the third period is 38 ms.

(First control flow)
The parking control part 36 performs the process of step SA1 of the 1st control flow shown in FIG. 13 for every 1st period.

  First, the generation unit 32 determines whether parking control has been started (step SA1). Specifically, parking control is started by detecting a signal when the parking control unit 36 executes “garage parking control” in response to the user's operation, and the generation unit 32 starts the control. Determine whether or not.

  Next, the generation unit 32 executes the generation process described above (step SA2). Thereby, the production | generation part 32 produces | generates the synthesized image which looked down at the vehicle 1 virtually.

  Next, the display unit 33 executes the display process described above (step SA3). As a result, the display unit 33 displays the composite image on the display 12.

  Next, the parking control system SY ends the first control flow shown in FIG.

  That is, since the parking control system SY performs “garage parking control” in response to the user's operation, the composite image temporally generated on the display 12 during the “garage parking control” is executed in real time. indicate.

  For this reason, the parking control system SY can display in real time the motion of the vehicle 1 on which “garage parking control” is performed on the display 12. As a result, the user can see in real time the vehicle 1 for which garage parking control is performed.

(Second control flow)
The parking control system SY performs the process of step SB1 of the second control flow shown in FIG. 14 every second period.

  First, the acquisition part 20 acquires each vehicle exterior image which the cameras 3-6 image | photographed (step SB1). Then, the acquisition unit 20 transmits a plurality of outside-vehicle images to the image processing unit 21.

  Next, the recognizing unit 30 executes the above-described recognition process (step SB2). Thereby, the image of the parking frame included in the plurality of images outside the vehicle is recognized.

  Next, the selection part 31 performs the selection process mentioned above (step SB3). Thereby, the image of the parking frame with the highest recognition accuracy is selected from the images of the parking frame recognized in each of the plurality of images outside the vehicle.

  Therefore, since the parking control system SY performs the second parking control executed in the third control flow described later based on the image of the parking frame with the highest recognition accuracy, the vehicle 1 is appropriately set to the parking position PP. It can be parked.

(Third control flow)
Parking control system SY performs the process of step SC1 of the 3rd control flow shown in Drawing 15 for every 3rd period.

  First, the parking control unit 36 determines whether or not to start control in which the vehicle 1 moves backward from the reverse start position P3 to the parking position PP shown in FIG. 6 to park (step SC1).

  If it is not determined to start the control for moving the vehicle 1 backward and parking (NO in step SC1), the parking control unit 36 executes the first parking control process described above (step SC2). That is, the parking controller 36 controls the vehicle 1 from the initial position P1 shown in FIG. 6 so as to travel in parallel with a plurality of parking frames positioned in parallel.

  Next, the derivation unit 34 performs the basic derivation process described above (step SC3). That is, when the vehicle 1 moves from the initial position P1 shown in FIG. 6 to the reverse start position P3, the recognition unit 30 recognizes a plurality of parking frame images.

  Next, the third control flow is terminated.

  And parking control system SY performs processing of Step SC1 of the 3rd control flow in the next cycle.

  When it is determined by the parking control unit 36 that it is determined that the control for causing the vehicle 1 to move backward and park is started (YES in step SC1), the vehicle 1 is moved backward and parked. (When NO in step SC4), the derivation unit 34 executes the first derivation process described above (step SC5).

  In addition, when the control which reversely moves the vehicle 1 and starts parking and the control which causes the vehicle 1 to reversely move and park is not being executed, the state immediately before the control where the vehicle 1 is moved backward and parked is performed. It is.

  Next, the derivation unit 34 executes the second derivation process described above (step SC6). That is, the deriving unit 34 derives a route R on the virtual map that guides the vehicle 1 into the parking frame based on the derived reverse start position P3 and the position of the parking frame.

  Next, the third control flow is terminated.

  And parking control system SY performs processing of Step SC1 of the 3rd control flow in the next cycle.

  When it is determined by the parking control unit 36 that the control for moving the vehicle 1 backward and parking is performed when it is determined that the control for moving the vehicle 1 backward and parking is started (YES in step SC1). (When YES in step SC4), determination unit 35 executes the determination process described above (step SC7). That is, after the control for moving the vehicle 1 backward by the “garage parking control” is started, the determination unit 35 determines the current position of the vehicle 1 on the virtual map based on the recognized image of the parking frame.

  Next, the parking control part 36 performs the 2nd parking control process mentioned above (step SC8). That is, the behavior of the vehicle is controlled so that the current position of the vehicle 1 on the virtual map matches the route R on the virtual map.

  Next, the third control flow is terminated.

  That is, by executing the parking control in which the vehicle 1 travels and parks in the parking frame so that the current position of the vehicle 1 on the virtual map matches the route R on the virtual map, that is, by performing feedback control. Even if the current position of the vehicle 1 deviates from the route R on the virtual map, it can be returned to the route R. As a result, the vehicle 1 can be appropriately parked at the parking position PP.

  Furthermore, the parking control system SY performs the second parking control based on the recognition process executed in the second control flow described above and the image of the parking frame with the highest recognition accuracy selected by the selection process. The vehicle 1 can be parked at the parking position PP with high accuracy.

<Second Embodiment>
Next, a second embodiment will be described. In the following description, differences between the first embodiment and the second embodiment will be mainly described.

  The selection process of the first embodiment is that “the accuracy of recognition among the images of the parking frame recognized by the recognition unit 30 based on the plurality of outside images obtained by the plurality of cameras mounted on the vehicle is the highest. Although the selection unit 31 selects a large parking frame image, the selection process of the second embodiment is performed when the vehicle 1 is moving backward in a curved line by the garage parking control. The selection unit 31 selects a parking frame image included in the acquired outside-vehicle image.

  Details of the selection process will be described.

  By the first parking control process executed by the parking control unit 36, the vehicle 1 is temporarily stopped at the reverse start position P3 as shown in FIG. The direction in which the vehicle 1 stops is such that the rear portion of the vehicle 1 faces the vehicle entrance side of the parking frame. And the direction turns so that the front-back direction of the vehicle 1 and the long side direction of a parking frame may become a predetermined angle (for example, 25 degree | times).

  For this reason, the route R on the virtual map derived by the second derivation process executed by the derivation unit 34 becomes a curve.

  That is, the backward movement of the vehicle 1 by the second parking control process executed by the parking control unit 36 is curved.

  Then, as shown in FIG. 8, when the parking control unit 36 moves backward so that the current position of the vehicle 1 on the virtual map matches the route R on the virtual map, that is, the vehicle 1 is curved in the left direction. When moving backward, the right side camera 5 of the vehicle 1 is closest to the parking frame.

  As a result, when the vehicle 1 is moving backward in a curve in the left direction, the accuracy of recognizing the parking frame included in the vehicle outside image captured by the right side camera 5 is the parking frame included in the vehicle outside image captured by another camera. It tends to be the largest compared to the accuracy of recognition.

  Therefore, when the vehicle 1 is moving backward in a curved line by the garage parking control, the selection unit 31 selects the image of the parking frame included in the outside image acquired by the side camera, so that the probability of recognition is the highest. A parking frame is selected. As a result, since the second parking control is executed based on the image of the parking frame with the highest recognition accuracy, the vehicle 1 can be appropriately parked at the parking position PP.

<Third Embodiment>
Next, a third embodiment will be described. In the following description, differences between the first embodiment and the third embodiment will be mainly described.

  The selection process of the first embodiment is that “the accuracy of recognition among the images of the parking frame recognized by the recognition unit 30 based on the plurality of outside images obtained by the plurality of cameras mounted on the vehicle is the highest. The selection unit 31 selects an image of a large parking frame.

  In the third embodiment, the selection process is omitted, and a plurality of parking frame images recognized by the recognition unit 30 are recognized based on each of a plurality of outside images obtained by a plurality of cameras mounted on the vehicle. The calculation unit executes a calculation process for converting the accuracy into a value and making the value a relative value (weight).

  As described above, the accuracy is “a certainty in the case where the image length or thickness of the object grasped by the edge detected by the recognition unit 30 is the length or thickness of the image of the parking frame. The reason why the probability is relatively large is, for example, the case where the edge length is relatively long without interruption in the region below the center of the vehicle outside image, or the case where the edge thickness is relatively thick. Explained.

  Based on such a criterion, the calculation unit quantifies the accuracy of recognizing each parking frame image, and sets the relative values so that the sum of the respective values (weights) becomes 100.

  Then, the deriving unit 34 derives an appropriate angular position by weighting each angular position with each value.

  For example, on the virtual map of the image of the parking frame included in the image outside the vehicle photographed by the side camera 5, that is, the corner position on the XY coordinates is X = 5, Y = 5, and the image outside the vehicle photographed by the rear camera 6 The angular positions on the XY coordinates of the included parking frame image are X = 5 and Y = 8. The former angular position value (weight) is 60, and the latter angular position value (weight) is 40.

  In such a case, the deriving unit 34 derives an angular position consisting of X = 5 and Y = 6.2 as a result of weighting each angular position with each value.

  Thereby, parking control system SY can perform parking control based on the image of the parking frame derived appropriately. As a result, the parking control system SY can appropriately park the vehicle 1 at the parking position.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications are possible. Below, the modification which is other embodiment is demonstrated. Of course, you may combine the form demonstrated below suitably.

<Modification 1>
In the above representative embodiment, the side line has been described as “a long and thin U-shaped white line”, but it may be a T-shaped white line as shown in FIG.

  Alternatively, a U-shaped white line may be used. Alternatively, it may be an I-shaped white line.

  This is because the shape can recognize the angular position of the side line. Accordingly, the shape of the side line is not limited to this as long as the corner position can be recognized.

DESCRIPTION OF SYMBOLS 1 Vehicle 2 Image processing apparatus 3 Front camera 4 Left side camera 5 Right side camera 6 Rear camera 13 Parking control apparatus 20 Acquisition part 21 Image processing part 22 Control part 23 Output part 30 Recognition part 31 Selection part 32 Generation part 33 Display part 34 Derivation Part 35 determination part 36 parking control part

Claims (9)

A parking control system that is mounted on a vehicle and performs parking control to control the behavior of the vehicle to park the vehicle in a parking frame,
Obtaining means for obtaining a plurality of outside images obtained by a plurality of cameras mounted on the vehicle;
Recognizing means for recognizing the image of the parking frame included in the plurality of outside-vehicle images;
Selecting means for selecting the image of the parking frame having the highest recognition accuracy among the images of the parking frame recognized in the plurality of images outside the vehicle;
Control means for executing the parking control based on the image of the selected parking frame;
A parking control system comprising: In the parking control system according to claim 1,
The recognizing means detects an edge of the image of the parking frame in the outside image;
The parking control system characterized in that the selection means obtains the recognition accuracy based on the edge. In the parking control system according to claim 1 or 2,
The plurality of cameras includes a side camera that captures a side of the vehicle,
The said selection means selects the image of the said parking frame contained in the said image outside the vehicle which the said side camera acquired, when the said vehicle is going backwards by the said parking control, The parking control system characterized by the above-mentioned . In the parking control system according to any one of claims 1 to 3,
The plurality of cameras includes a rear camera that captures the rear of the vehicle,
The parking control system, wherein the selection means selects an image of the parking frame included in the outside image acquired by the rear camera when starting reverse driving of the vehicle by the parking control. In the parking control system according to any one of claims 1 to 4,
First derivation means for deriving a reverse start position of the vehicle and a position of the parking frame on a virtual map based on the recognized image of the parking frame before starting the reverse of the vehicle by the parking control; ,
Second derivation means for deriving a route on the virtual map for guiding the vehicle into the parking frame based on the derived reverse start position and the position of the parking frame;
Determination means for determining the current position of the vehicle on the virtual map based on the recognized image of the parking frame after starting the reverse of the vehicle by the parking control;
With
The said control means controls the behavior of the said vehicle so that the said current position may correspond with the said route, The parking control system characterized by the above-mentioned. In the parking control system according to any one of claims 1 to 5,
A parking control system, wherein the lenses of the plurality of cameras are fish-eye lenses. In the parking control system according to any one of claims 1 to 6,
Generating means for generating a composite image in which the periphery of the vehicle is viewed from a virtual viewpoint based on the plurality of outside-vehicle images;
Display means for displaying the composite image during execution of the parking control;
A parking control system further comprising: A parking control method for performing parking control for controlling the behavior of the vehicle to park the vehicle in a parking frame,
(A) obtaining a plurality of outside-vehicle images obtained by a plurality of cameras mounted on the vehicle;
(B) recognizing an image of the parking frame included in the plurality of images outside the vehicle;
(C) selecting the image of the parking frame having the highest recognition accuracy among the images of the parking frame recognized in the plurality of images outside the vehicle;
(D) executing the parking control based on the selected image of the parking frame;
A parking control method comprising: An image processing apparatus for processing an image,
Acquisition means for acquiring a plurality of outside images obtained by a plurality of cameras mounted on the vehicle;
Recognizing means for recognizing the image of the parking frame included in the plurality of outside-vehicle images;
Selecting means for selecting the one with the highest recognition accuracy among the images of the parking frame recognized in each of the plurality of outside images;
Output means for outputting information based on the image of the selected parking frame to a parking control device that controls the behavior of the vehicle and parks the vehicle in the parking frame;
An image processing apparatus comprising:

Images