JP2017076275A - Parking section recognition device and parking section recognition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate a specification of a parking section, and improve recognition accuracy of a parking section line.SOLUTION: A broad recognition unit 22 is configured to conduct detection processing of detecting candidate lines from a broad image area, and recognize each parking section line on the basis of a pair of detected candidate lines to specify a parking section upon a forward movement of a vehicle. A forward movement tracking unit 32 is configured to, upon the forward movement of the vehicle, use each parking section line recognized by the broad recognition unit 22 as an object section line to track a location of each object section line on the basis of a recognition result recognized by the broad recognition unit 22. A rearward movement tracking unit 34 is configured to, upon rearward movement of the vehicle, track the location of each object section line on the basis of a tracking result by the forward movement tracking unit 32 and behavior information on the vehicle. An area setting unit 40 is configured to set a narrow image area on the basis of the location of each object section line recognized by the broad recognition unit 22. When detecting one candidate line from the narrow image area set by the area setting unit 40, a narrow recognition unit 24 is configured to recognize each object section line on the basis of the detected one candidate line and the tracking result by the rearward movement tracking unit 34.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a technique for recognizing a parking lot line using a captured image of an in-vehicle camera and specifying the parking lot.

  Conventionally, in a parking support system that supports driving operation at the time of parking, each parking lot line is recognized by detecting a pair of candidate lines that are candidates for a parking lot line based on a captured image of an in-vehicle camera. A technique for identifying a parking lot from a parking lot line is known. In the parking assistance system, parking assistance control such as guide image display control and vehicle drive control is performed so that the vehicle fits within the frame of the parking section identified using this type of technology.

  In such parking area recognition technology, based on the previous recognition result and the dead reckoning result on the behavior of the vehicle, the position of each parking line on the captured image in the current recognition is predicted, and based on this prediction result, There has been proposed a technique for recognizing a candidate line (and thus a parking lot line) that could not be detected from a captured image this time due to a break or part of the parking lot line (see Patent Document 1).

JP, 2014-106731, A

  However, in the conventional technology, when the vehicle is moved backward and parked in the parking space, the parking line is recognized based on the captured image obtained when the vehicle is moving backward. For example, the vehicle is parked with the vehicle when starting backward. For example, when there is a parking section in a blind spot as viewed from the in-vehicle camera, such as a scene in which another vehicle is parked between the sections, there are the following problems. That is, in the above scene, a pair of candidate lines cannot be detected unless the vehicle is considerably close to the parking area, so it takes time to recognize the parking area line, and it is difficult to identify the parking area. was there. As a result, for example, the start timing of parking support control may be delayed, which may impair the convenience of parking support.

  In the prior art, the image processing area for detecting the candidate line in the photographed image is fixed. Therefore, if it is attempted to increase the recognition accuracy of the parking lot line by increasing the detection accuracy of the candidate line, the image processing burden is increased. There was also a problem that it was easily increased.

  The present invention has been made in view of such problems, and makes it easy to identify a parking area even when the parking area exists in a blind spot as viewed from the in-vehicle camera at the start of parking, and the parking line The object is to provide a technique capable of improving the recognition accuracy.

  A parking area recognition device according to one aspect of the present invention includes an image acquisition unit, a behavior information acquisition unit, a wide area recognition unit, a forward tracking unit, a backward tracking unit, an area setting unit, and a wide area recognition unit. . An image acquisition part acquires the picked-up image of the vehicle-mounted camera mounted in a vehicle and image | photographs the periphery of a vehicle. The behavior information acquisition unit acquires behavior information indicating the behavior of the vehicle.

  The wide area recognition unit performs a detection process of detecting candidate lines that are candidates for the parking lot lines from a predetermined wide area image area in the captured image acquired by the image acquisition unit, and a pair of detected by the detection process Each parking lot line is recognized based on a plurality of candidate lines including the candidate line, and a parking lot configured to include each recognized parking lot line is specified when the vehicle moves forward.

  The forward tracking unit sets each parking lot line constituting the parking lot specified by the wide area recognition unit as a target division line when the vehicle moves forward, and each target division line based on the recognition result by at least the wide area recognition unit. Keep track of the location. On the other hand, the backward tracking unit tracks the position of each target lane line based on at least the tracking result of the forward tracking unit and the behavior information acquired by the behavior information acquisition unit when the vehicle is moving backward.

  The area setting unit sets a narrow area image area that is at least smaller than the wide area image area in the captured image acquired by the image acquisition section based on the position of each target marking line recognized by the wide area recognition section. Then, the narrow area recognition unit performs the above-described detection process from the narrow area image region set by the area setting unit, and at least one candidate line detected by the detection process and the tracking result by the backward tracking unit Based on the above, each target lane line is recognized, and a parking lane including the recognized parking lane lines is specified when the vehicle is going backward.

  According to such a configuration, before reversing the vehicle to park in the parking space, the parking space is recognized in advance by detecting a pair of candidate lines from the captured image when the vehicle moves forward, Furthermore, since the position of the parking lot line is tracked based on the behavior of the vehicle, the parking lot line can be recognized based on the tracking result even when the pair of candidate lines cannot be detected when the vehicle is going backward. It becomes like this.

  Further, when the vehicle is moving backward, at least one candidate line is detected from the captured image, and the parking lot line is recognized based on the detected at least one candidate line and the tracking result described above. The parking lot line can be recognized without detecting the candidate line, and the deviation between the tracking result and the actual parking lot line position can be corrected by at least one candidate line detected from the captured image. become. Thereby, it becomes possible to advance the start timing of parking assistance control, maintaining the recognition accuracy of a parking lot line.

  Also, when the vehicle is moving backward, if the position of the parking lot line has already been detected by a pair of candidate lines, the image processing area is reduced based on the position of the parking lot line, so that the detection accuracy of candidate lines is improved. Even if it is increased, the processing load is offset by the narrowing of the image processing area, and the image processing load can be alleviated.

  Therefore, according to one aspect of the present invention, it is easy to specify the parking area even when the parking area exists in the blind spot viewed from the in-vehicle camera at the start of parking, and the recognition accuracy of the parking area line is improved. It can be made easy.

  Moreover, according to the parking area recognition method which is one aspect of the present invention, the same effects as those already described in the parking area recognition apparatus which is one aspect of the present invention can be obtained for the same reason as described above.

1 is a block diagram illustrating an overall configuration of a parking assistance system 100 including a parking section recognition device 1. FIG. 2 is a block diagram showing a functional configuration of a recognition control unit 10. FIG. 3 is a block diagram showing a functional configuration of a parking section recognition unit 16. FIG. It is a schematic diagram for demonstrating a wide area | region image area and a narrow area image area. It is a schematic diagram which shows the 1st scene which reversely drives a vehicle and parks in the space of a parking area. It is a schematic diagram which shows a mode when a vehicle is advanced to the position at the time of a reverse start in the 1st scene shown in FIG. It is a schematic diagram which shows the 2nd scene which reverses a vehicle and parks in the space of a parking area. It is a schematic diagram which shows the structural example of a parking area in the 2nd scene shown in FIG. It is a flowchart of a recognition control process. It is a flowchart of a wide area recognition process. It is a flowchart of a narrow area recognition process.

Hereinafter, embodiments for carrying out the invention will be described with reference to the drawings.
[1. First Embodiment]
[1-1. overall structure]
A parking section recognition device 1 shown in FIG. 1 includes a front camera 2, a rear camera 4, and a recognition control unit 10. Further, the parking section recognition device 1 is connected to an in-vehicle local area network (hereinafter referred to as an in-vehicle LAN 8) via a parking assistance ECU 50. The in-vehicle LAN 8 is connected with a parking assist ECU 50, an engine ECU 60, a steering ECU 70, a brake ECU 80, an HMI-ECU 90, and the like. And each control object and each sensor which are not illustrated corresponding to each ECU50-90 are connected, respectively. Hereinafter, a vehicle equipped with a parking assistance system 100 including these components is referred to as a host vehicle.

  Each ECU 50 to 90 and each control object and each sensor corresponding to each ECU 50 to 90 constitute an in-vehicle control system (not shown) in addition to the parking support system 100. Each in-vehicle control system is a well-known network system that is pre-installed in the host vehicle, and performs engine control, steering control, brake control, human machine interface (hereinafter, HMI) control, and the like by each ECU 50-90. System.

  Each control object is a component controlled by each of the ECUs 50 to 90. For example, an ignition mechanism, a fuel system, an intake / exhaust system, a valve mechanism, a start mechanism, a steering pump, a motor, and a brake are included in the engine. There are actuators that constitute the HMI, display devices and acoustic devices that constitute the HMI. Each sensor detects information necessary for control of each ECU 50 to 90. For example, the accelerator pedal depression amount, the steering steering amount, the brake pedal depression amount, various switch operation states, the speed / acceleration of the host vehicle, Some of them detect the steering angle, yaw rate, shift lever position, etc.

  Each of the ECUs 50 to 90 is configured around a well-known microcomputer or a communication controller for an in-vehicle network, and based on detection information acquired from each sensor and vehicle information received from the in-vehicle LAN 8, a control object assigned in advance It controls the drive or output. The vehicle information is information shared between the ECUs 50 to 90 in order to optimize the entire in-vehicle control system. For example, the detection information of each sensor described above, the driving of the control target to the ECUs 50 to 90, or There is command information for output.

  For example, in the parking support system 100, the engine ECU 60 controls the host vehicle by engine control so that the host vehicle fits within the frame of the designated parking section according to the command information received from the parking support ECU 50 via the in-vehicle LAN 8. The vehicle can be moved backward, the steering ECU 70 can steer the host vehicle by steering control, and the brake ECU 80 can be stopped while decelerating the host vehicle by brake control. Note that at least one of the engine control and the brake control here may be performed based on the operation of the driver.

  In addition, for example, in the parking support system 100, one HMI-ECU 90 operates one parking section from one or a plurality of parking sections by performing HMI control according to the command information received from the parking support ECU 50 via the in-vehicle LAN 8. A start image for the driver to specify and a guide image indicating a guide line that is a reference for steering operation for the driver are displayed so that the host vehicle fits within the frame of the specified parking section. In addition, the guide line here displays the block frame line that displays the frame of the designated parking block, the distance guide line that displays the guideline of the distance between the specified parking block and the vehicle, and the vehicle width of the vehicle Any of a vehicle width extension line to be displayed, an expected course line for displaying the expected course of the host vehicle, or a line appropriately combining these may be used.

  The parking assist ECU 50 is connected to the recognition control unit 10 and detects that the shift lever of the host vehicle has been switched to the reverse position. Information is acquired, based on the acquired specific section information, command information for displaying the above-described start image is generated and transmitted to the HMI-ECU 90.

  In addition, when the parking assistance ECU 50 receives operation information related to the parking area designated by the driver based on the start image, the parking assistance ECU 50 generates command information for displaying the above-described guidance image based on the received operation information, and generates an HMI. -It transmits to ECU90. Further, while detecting the position of a parked vehicle or the like around the vehicle using an ultrasonic sonar (not shown) or the like, the course of the own vehicle that can be parked so as not to hit such an obstacle is calculated, and the own vehicle along the calculated course is calculated. It is also possible to generate command information for reversing the engine and transmit it to the ECUs 60-80.

  The in-vehicle LAN 8 is a local area network provided in the host vehicle. For example, various in-vehicle information is transmitted using a communication protocol such as a well-known CAN, FlexRay, LIN, MOST, or AVC-LAN. To do. A display device (not shown) is installed at a position where a driver can easily view a display image, and is a liquid crystal display, a head-up display, an electronic mirror, or a combination thereof as appropriate. An acoustic device (not shown) is used, for example, to output an announcement indicating the start of parking assistance, or to output a warning sound or a warning message when the vehicle is likely to hit an obstacle.

  The front camera 2 and the rear camera 4 correspond to an in-vehicle camera that is mounted on the host vehicle and photographs the periphery of the host vehicle. For example, the lens is not exposed to wind and rain, and the driver's view is obstructed. In order to prevent this, it is fixed in the passenger compartment by bonding to the upper end of the windshield. In this case, the front camera 2 is provided on the front window, and the rear camera 4 is provided on the rear window.

  Further, the front camera 2 and the rear camera 4 have known imaging elements such as CMOS and CCD, for example, have an optical axis slightly horizontally downward toward the front of the host vehicle and the rear of the host vehicle, and spread in a predetermined angular range. Shoot the area. Light incident from such an imaging region is photoelectrically converted by an image sensor, and a signal read out as a stored charge voltage is amplified and converted into a digital image having a predetermined luminance gradation by A / D conversion. Hereinafter, a digital image generated by the front camera 2 is referred to as a front image, and a digital image generated by the rear camera 4 is referred to as a rear image.

[1-2. Configuration of recognition control unit 10]
The recognition control unit 10 is composed of a known microcomputer having a CPU 5 and a semiconductor memory (hereinafter referred to as a memory 6) such as a RAM, a ROM, and a flash memory, and a communication controller for an in-vehicle network, and is stored in the memory 6. Various processes are executed by the CPU 5 based on the programmed program. That is, by executing this program, a method corresponding to the program is executed. In the recognition control unit 10, the number of microcomputers may be one or plural, and each installation location of the one or more microcomputers may be any inside the vehicle.

  As shown in FIG. 2, the recognition control unit 10 is configured as a function realized by executing various processes of the CPU 5. As shown in FIG. 2, the image acquisition unit 12, the behavior information acquisition unit 14, the parking section recognition unit 16, and the parking section And an output unit 18. Note that some or all of these functions provided by the recognition control unit 10 may be configured by hardware using one or a plurality of electronic circuits such as logic circuits and ICs. That is, the recognition control unit 10 can provide the above function not only by software but also by hardware or a combination thereof.

  The image acquisition unit 12 acquires a front image and a rear image generated by the front camera 2 and the rear camera 4, respectively. In the present embodiment, both the front image and the rear image are acquired when the host vehicle is moving forward, such as when the shift lever of the host vehicle is in the drive position, and when the shift lever of the host vehicle is in the reverse position, that is, the host vehicle is moving backward. Sometimes a back image is acquired. The captured image acquired in this way is output to the parking section recognition unit 16. Note that the front image may be acquired when the host vehicle is moving forward, and the rear image may be acquired when the host vehicle is moving backward.

  The behavior information acquisition unit 12 acquires behavior information indicating the behavior of the host vehicle. The behavior information in the present embodiment is the detection information of the speed, acceleration, steering angle, yaw rate, etc. of the host vehicle and the shift lever position, and is received from the other ECUs 60 to 80 from the in-vehicle LAN 8 via the parking assist ECU 50, for example. Can do. The behavior information acquired in this way is output to the parking section recognition unit 16. The behavior here means a basic motion such as stop, acceleration, deceleration, constant speed, etc. plus an additional motion such as yawing.

  The parking area output unit 18 transmits information on one or more parking areas specified by the parking area recognition unit 16 to the parking assist ECU 50 via the in-vehicle LAN 8 as specific area information described later.

[1-3. Configuration of parking section recognition unit 16]
As shown in FIG. 3, the parking section recognition unit 16 includes an image processing unit 20, a target tracking unit 30, and an area setting unit 40.

  The image processing unit 20 processes the captured image acquired by the image acquisition unit 12 to detect a candidate line that is a candidate for a parking lot line from the captured image. Specifically, the image processing unit 20 detects an edge by searching an area having a luminance equal to or higher than a predetermined threshold value from the bottom of the predetermined image processing area in the frame based on the luminance of the image data, for example. To do. Since the white line has edges that are high-frequency components at both ends, if the luminance value of the image data is differentiated in the horizontal direction, peaks are obtained at both ends of the white line. A pixel having a horizontal luminance gradient or difference of a predetermined value or more is an edge.

  If this edge is connected in the vertical direction of the photographed image, the white line portion can be estimated, and the white line is estimated by applying a matching method to the estimated white line portion based on features such as the threshold of the white line width and the linear shape. decide. This feature is quantitatively evaluated as an index indicating the likelihood of white lines and stored in the memory 6. Note that, by variably setting the luminance weight of image data for each pixel color such as RGB, candidate lines of other colors such as a yellow line and a blue line can be determined by the same method as in the case of a white line. .

  A model expression of the candidate line is obtained by extracting a plurality of edges of the candidate line such as the white line determined in this way and using, for example, the Hough transform or the minimum median method (hereinafter, LMedS). This model formula indicates the position of the candidate line with respect to the host vehicle, and includes information such as the length, width, and curvature of the candidate line. In this way, candidate line information indicated by the model expression of each candidate line is obtained by the detection processing of the image processing unit 20.

  In addition, the image processing unit 20 is narrower than the wide area recognition unit 22 due to the difference in the size of the image processing area described above when performing the detection process for detecting the candidate line and the difference in the recognition method of the parking lot line. The area recognition unit 24 is roughly divided. Specifically, as shown in FIG. 4, the size of the image processing area is an image of a wide area image area that is determined in advance by the wide area recognition unit 22 as an area including a portion where a road surface is projected in a frame of a captured image, for example. The processing area is different from the processing area in that the narrow area recognition unit 24 sets a narrow area image area smaller than at least the wide area image area in the frame of the captured image as an image processing area. On the other hand, in the parking line recognition method, the wide area recognition unit 22 recognizes the parking line based on a plurality of candidate lines including a pair of candidate lines detected by the detection process, whereas the narrow area recognition unit 24 The difference is that the parking lot line is recognized based on at least one candidate line detected by the detection process and the tracking result of the parking lot line by the target tracking unit 30 as described later.

  The area setting unit 40 determines the likelihood of the parking lot line recognized by the wide area recognition unit 22 according to the likelihood that represents the likelihood that the pair of candidate lines detected by the wide area recognition unit 22 are estimated to be parking lot lines. By setting the narrow area image region based on the position, the function of the image processing unit 20 is switched from the wide area recognition unit 22 to the narrow area recognition unit 24. The narrow-area image area here is obtained by adding a predetermined margin in the vertical and horizontal directions to the length and width of the parking lot line recognized by the wide-area recognition unit 22, for example, as shown in FIG. Is set. Further, the “position of the parking lot line recognized by the wide area recognition unit 22” here is based on the position of the parking lot line recognized from the captured image in addition to the position of the parking lot line recognized from the captured image. The position of the parking lot tracked by the target tracking unit 30 is also included. Further, in the present embodiment, the “position of the parking lot line” refers to a relative position with respect to the position of the host vehicle. Further, the “position of the host vehicle” here refers to a reference position that is predetermined in the host vehicle, such as a rear center position of the host vehicle.

  The target tracking unit 30 has a function of tracking the position of each target lane line with each parking lane line constituting the parking lane identified by the wide area recognition unit 22 as a target lane line. From the difference in direction, the vehicle is roughly divided into a forward tracking unit 32 and a backward tracking unit 34. Specifically, for example, as shown in FIG. 5, when the host vehicle is moved backward and parked in the parking space, another vehicle is parked between the host vehicle and the parking space when starting reverse. In this case, the pair of candidate lines may not be detected unless the host vehicle is considerably close to the parking area.

  In order to deal with such scenes, the target tracking unit 30 detects a pair of candidate lines in advance when the host vehicle moves forward by the wide area recognition unit 22 before moving the host vehicle backward as shown in FIG. The recognized parking lane line is set as the target lane line, and the forward tracking unit 32 tracks the position of each target lane line based on at least the recognition result by the wide area recognition unit 22. On the other hand, when the shift lever of the host vehicle is switched to the reverse position, the backward tracking unit 34 is based on at least the tracking result by the forward tracking unit 32 and the behavior information acquired by the behavior information acquiring unit 12. It also tracks the location of the lane markings.

  In the present embodiment, the forward tracking unit 32 is configured so that each target section constituting the parking section identified from the front image by the wide area recognition unit 22 when the shift lever of the own vehicle is at the drive position, that is, when the own vehicle is moving forward. The position of the line is stored in the memory 6, and when the position of each target partition line deviates from the imaging area of the front camera 2, each target partition is used using the dead reckoning result based on the behavior information acquired by the behavior information acquisition unit 12. The position of the line is calculated and stored in the memory 6. Further, when the position of each target lane line enters the imaging area of the rear camera 4, the position of each target lane line recognized from the rear image by the wide area recognition unit 22 is stored in the memory 6, and the position of each target lane line is determined. When the rear camera 4 is out of the shooting area, the dead reckoning result based on the behavior information acquired by the behavior information acquisition unit 12 is used again to calculate the position of each target lane line and store it in the memory 6. May be. Thus, the forward tracking unit 32 also tracks the position of each target lane marking that is out of the imaging region of the in-vehicle camera such as the front camera 2 or the rear camera 4. Here, “dead reckoning” is a self-position estimation method that uses a combination of the speed, acceleration, steering angle, yaw rate, and the like of the host vehicle to estimate the relative position of the host vehicle with respect to each target lane marking. For this reason, by using the dead reckoning result, it is possible to estimate the relative position of each target marking line with respect to the host vehicle.

  On the other hand, when the shift lever of the host vehicle is switched to the reverse position, the backward tracking unit 34 takes over the result of tracking the position of each target lane line by the forward tracking unit 32, and the wide area recognition unit 22 or the narrow area recognition unit 24. To track the position of each target marking line recognized from the captured image. Specifically, the backward tracking unit 34 is used when the shift lever of the host vehicle is in the reverse position, that is, when the position of each target lane line is out of the imaging area of the rear camera 4 when the host vehicle is moving backward. Calculates the position of each target lane line using the dead reckoning result based on the behavior information acquired by the behavior information acquisition unit 12 with the position of each target lane line tracked by the forward tracking unit 32 as a starting point. And stored in the memory 6. In addition, when the position of each target lane line is within the imaging area of the rear camera 4, each of the target lane lines constituting the parking area identified from the rear image by the wide area recognition unit 22 or the narrow area recognition unit 24 is displayed. The position is stored in the memory 6. Further, when the position of each target lane line deviates from the shooting area of the rear camera 4, the position of each target lane line is determined using the dead reckoning result described above, starting from the position of the target lane line recognized from the rear image immediately before. Is calculated and stored in the memory 6. In this way, the backward tracking unit 34 also tracks the position of each target lane line that is out of the imaging area of the rear camera 4.

  In this way, for example, when the host vehicle moves backward in order to avoid a collision with another vehicle parked on the near side with respect to the position of the host vehicle when moving backward, Even if the position may deviate from the shooting area of the rear camera 4, for example, if at least one candidate line corresponding to the parking lot line on the back side with respect to the position of the host vehicle can be detected, the parking lot line 2 is formed. The parking lot line of the book can be recognized.

  The scene where the parking area is present in the blind spot as viewed from the in-vehicle camera is not limited to the above-described scenes shown in FIGS. 5 and 6. For example, as shown in FIG. When the vehicle is parked in the parking space, a scene in which the host vehicle at the start of reverse driving is located quite close to the parking space is also assumed. In such a scene, a pair of candidate lines may not be detected unless the vehicle is once moved away from the parking section. On the other hand, similarly, for example, three parking lot lines constituting one parking lot are recognized in advance by detecting a pair of candidate lines from the front image when the host vehicle moves forward, and further the behavior of the host vehicle. By tracking the positions of the three parking lot lines based on the vehicle, for example, at least one candidate line corresponding to the parking lot line on the back side with respect to the position of the own vehicle is detected when the host vehicle moves backward If possible, the remaining two (that is, a total of three) parking lot lines constituting the parking lot line can be recognized. By the way, as shown in FIG. 8, similarly for a parking space for parallel parking constituted by two parking lot lines, if at least one candidate line corresponding to the rear parking lot line can be detected, A total of two parking lot lines constituting the parking lot line can be recognized.

[1-4. processing]
[1-4-1. Recognition control process]
Next, processing executed by the CPU 5 of the recognition control unit 10 (hereinafter, recognition control processing) will be described with reference to the flowchart of FIG. This process is started when the vehicle speed becomes equal to or lower than a reference value, for example, and ends when the vehicle speed exceeds the reference value or the shift lever of the host vehicle is switched to the parking position.

  When this process is started, first, in S110, the CPU 5 determines, based on the behavior information acquired by the behavior information acquisition unit 14, whether the shift lever of the host vehicle is in the drive position, that is, whether the host vehicle is moving forward. . If it is determined that the host vehicle is moving forward, the process proceeds to S120. If it is determined that the host vehicle is not moving forward, the process proceeds to S140.

  Next, in S120, the CPU 5 performs a process of recognizing a parking lot line (hereinafter, a wide area recognition process) with the function in the image processing unit 20 as the wide area recognition unit 22 by the area setting unit 40 setting the wide area image area. Start. In the wide area recognition process here, a detection process for detecting a candidate line that is a candidate for a parking lot line from a predetermined wide area image area in a front image is performed, and a pair of candidate lines detected by this detection process is detected. A parking lot line is recognized based on a plurality of candidate lines included, and a parking lot including each recognized parking lot line is specified.

  In S130, with the start of the wide area recognition process in S120, the CPU 5 starts processing for tracking the position of the target lane line using the function in the target tracking unit 30 as the forward tracking unit 32 (hereinafter referred to as forward tracking processing). To do. In the forward tracking process here, each parking lot line constituting the parking lot identified by the wide area recognition process in S120 is set as a target division line, and the position of each target division line is tracked by the method described above. .

  Next, in S140, based on the behavior information acquired by the behavior information acquisition unit 14, the CPU 5 determines whether the shift lever of the host vehicle is in the reverse position, that is, whether the host vehicle is moving backward. If it is determined that the host vehicle is moving backward, the process proceeds to S150. If it is determined that the host vehicle is not moving backward, the process proceeds to S110.

  In S150, the CPU 5 cancels the forward tracking process in accordance with the shift lever position being switched to the reverse position in S140, and sets the position of the target lane line as the backward tracking unit 34 using the function in the target tracking unit 30. The tracking process (hereinafter referred to as backward tracking process) is started. In the backward tracking process here, the tracking result of the position of each target lane line by the forward tracking process of S130 is taken over.

  In subsequent S160, CPU 5 has a likelihood that represents the likelihood that the pair of candidate lines detected by the wide area recognition process in S120 or S170, which will be described later, being a parking lot line is higher than a predetermined threshold value. Judge whether or not. If it is determined that the likelihood is higher than the threshold, the process proceeds to S180, and if it is determined that the likelihood is equal to or less than the threshold, the process proceeds to S170, and S160 is continued while continuing the wide area recognition process. In the wide area recognition process here, a detection process for detecting a candidate line that is a candidate for a parking lot line from a predetermined wide area image area in the rear image is performed, and a pair of candidate lines detected by this detection process is detected. A parking lot line is recognized based on a plurality of candidate lines included, and a parking lot including each recognized parking lot line is specified. The likelihood here is quantitatively calculated by the wide area recognition process of S120 or S170 as will be described later.

  Next, in S180, the CPU 5 cancels the wide area recognition process in S120 or S170 described later, and the area setting unit 40 sets the narrow area image area, whereby the function in the image processing unit 20 is set as the narrow area recognition unit 24. A process for recognizing a parking lot line (hereinafter, narrow area recognition process) is started. In the narrow area recognition process here, a detection process for detecting a candidate line that is a candidate for a parking lot line from the narrow area image area set in the rear image by the area setting unit 40 is performed, and is detected by this detection process. Further, each target lane line is recognized based on at least one candidate line and a tracking result obtained by the backward tracking process, and a parking area including each recognized target lane line is specified.

  In S180, the CPU 5 uses the information indicating the position, length, width, and the like of each parking lot line constituting the parking lot specified by the narrow area recognition process in S180 as the specific zone information. By outputting, it transmits to parking assistance ECU50 via in-vehicle LAN8. The specific section information here is transmitted together with the identification information of the parking section every time the parking section is specified by the narrow area recognition processing of S180.

[1-4-2. Wide area recognition processing]
Next, the wide area recognition process executed by the CPU 5 in S120 and S170 will be described with reference to the flowchart of FIG.

  When this processing starts, first, in S210, the CPU 5 performs edge detection and white line extraction from the wide-area image region in the captured image acquired by the image acquisition unit 12 by the method described above, and calculates the model formula of the extracted white line. To do. Note that the well-known Hough transform is used for the calculation of the model formula here.

  In subsequent S220, the CPU 5 extracts a white line pair based on the candidate line information indicated by the model formula of each white line calculated in S210. The white line pair here corresponds to a pair of candidate lines. Further, in the extraction of the white line pair here, for example, a pair of white lines that are similar to each other are selected based on the width and curvature of each white line.

  Next, in S230, the CPU 5 calculates the above-described likelihood for the white line pair extracted in S220 based on at least one index of each length, interval, and parallelism. The likelihood here is larger as the length of each white line is closer, and the distance between one white line and the other white line is closer to a value obtained by adding a predetermined margin to the vehicle width of a general vehicle. The larger the value, the larger the angle formed by one white line and the other white line is closer to 0 °. Further, in the calculation of the likelihood here, the larger the index value indicating the white line-likeness of each white line, the larger the likelihood.

  In subsequent S240, the CPU 5 recognizes the parking lot line based on the likelihood calculated in S230. In this embodiment, the threshold value of S160 is set as the first threshold value, the likelihood is compared with a second threshold value set in advance as a value smaller than the first threshold value, and the likelihood is higher than the second threshold value. When it is high, the white line pair extracted in S220 is recognized as a parking lot line. In addition, for example, when there is a white line connecting the short portions of the pair of parking lot lines, the white line is also recognized as a parking lot line.

  In S250, the CPU 5 identifies the parking lot by each parking lot recognized in S240. In S260, the CPU 5 sets each parking lot constituting the parking lot as a target lot, and proceeds to S210.

[1-4-3. Narrow area recognition processing]
Next, the narrow area recognition process executed by the CPU 5 in S180 will be described using the flowchart of FIG.

  When this processing starts, first, in S310, the CPU 5 performs edge detection and white line extraction from the narrow area image region in the captured image acquired by the image acquisition unit 12 by the above-described method, and obtains the model formula of the extracted white line. calculate. Note that LMedS is used for calculation of the model formula here as a finer calculation method than the Hough transform of S210.

  In subsequent S320, the CPU 5 extracts at least one candidate line based on the candidate line information indicated by the model formula of each white line calculated in S310. In S330, it is determined whether or not a white line pair corresponding to a pair of candidate lines can be extracted as in S220. If a white line pair can be extracted, the process proceeds to S340 and the same as S230 to S250. A parking section is specified by a method (hereinafter, normal processing), and the process proceeds to S310. On the other hand, if the white line pair cannot be extracted, the process proceeds to S350.

  Next, in S350, the CPU 5 compares the position of each target lane line stored immediately before in the memory 6 by the backward tracking process in S150 and the position of one candidate line extracted in S320, so that S320 A white line pair is extracted by estimating the position of the other candidate line that forms a white line pair together with one candidate line extracted by the above. At this time, the position of each target lane line tracked by the backward tracking process is corrected using the position of one candidate line extracted in S320. And the white line pair extracted in this way is recognized as a parking lot line.

  In S360, the CPU 5 specifies a parking lot by the parking lot recognized in S250, sets each parking lot constituting the parking lot as a target lot in S370, and proceeds to S310.

[1-5. effect]
According to the first embodiment described in detail above, the following effects can be obtained.
(1a) Before moving the host vehicle backward, the parking lot line is recognized in advance by detecting a pair of candidate lines from the photographed image when the vehicle moves forward, and the position of the parking lot line based on the behavior of the host vehicle. Therefore, even if the pair of candidate lines cannot be detected when the vehicle moves backward, the parking lot line can be recognized based on the tracking result.

  (2a) When the vehicle moves backward, a candidate line is detected from the rear image, and the parking lot line is recognized based on the detected candidate line and the tracking result of the target lot line, so a pair of candidates The parking lot line can be recognized without detecting the line, and the deviation between the tracking result of the target parcel line and the actual position of the parking lot line can be corrected by one candidate line detected from the rear image.

  (3a) When the vehicle is moving backward, if the position of the parking lot line has already been detected by a pair of candidate lines, the image processing area is reduced based on the position of the parking lot line. Even if a fine calculation method is used, the processing load is offset by the narrowing of the image processing area, and the image processing load can be reduced.

  (4a) Therefore, according to the above (1a) to (3a), when the vehicle is moved backward in order to park in the parking space, there is a parking space at the blind spot as seen from the in-vehicle camera at the start of the backward movement. However, it is possible to easily identify the parking lot and to improve the recognition accuracy of the parking lot line.

  (5a) Further, in the backward tracking process, the tracking area is used to track the position of each target lane line recognized based on the rear image by taking over the result of tracking the position of each target lane line by the forward tracking process. The time until the line is recognized can be shortened. As a result, the start timing of the parking assistance control can be advanced, and the usability for the driver can be improved.

  (6a) Further, in the forward tracking process, since the position of each target partition line recognized based on the captured images of the front camera 2 and the rear camera 4 is tracked, the parking section is determined from either the front image or the rear image. If the line is recognized, the recognition result is reflected, and as a result, the tracking accuracy can be improved.

  (7a) In the backward tracking process, the position of each target lane line recognized by the wide area recognition process or the narrow area recognition process is tracked, so that the parking lane line can be recognized by the wide area recognition process when the host vehicle moves forward. Even if it is not, if the parking lot line is recognized by the wide area recognition process when the host vehicle is moving backward, the recognition result can be reflected.

  (8a) In the recognition control process, when the likelihood representing the likelihood that the pair of candidate lines detected by the wide area recognition process is a parking lot line is higher than the first threshold, the narrow area Since the image area is set, the recognition accuracy of the parking lot line in the narrow area recognition process can be improved.

  (9a) In the recognition control process, since the likelihood is calculated based on at least one of the length, the interval, and the parallelism of the pair of candidate lines, the likelihood of the parking lot line can be appropriately obtained.

[2. Other Embodiments]
As mentioned above, although the form for implementing this invention was demonstrated, this invention is not limited to the above-mentioned embodiment, It can implement in various deformation | transformation.

  (2A) In the above embodiment, in the forward tracking process, the position of each target lane line recognized based on the captured images of the front camera 2 and the rear camera 4 is tracked, and in the backward tracking process, the rear camera 4 Although the position of each target division line recognized based on the captured image is tracked, the present invention is not limited to this. For example, in the forward tracking process, the position of each target lane line recognized based on the captured image of the front camera 2 is tracked, and in the backward tracking process, each target recognized based on the captured image of the rear camera 4 The position of the lane marking may be tracked. By doing so, the processing burden can be reduced.

  (2B) The functions of one component in the above embodiment may be distributed as a plurality of components, or the functions of a plurality of components may be integrated into one component. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified only by the wording described in the claim are embodiment of this invention.

  (2C) In addition to the parking area recognition device 1 described above, the parking assistance system 100 having the parking area recognition device 1 as a constituent element, one or more programs for causing the computer to function as the parking area recognition device 1, this program The present invention can also be realized in various forms such as a non-transitional tangible recording medium such as one or a plurality of semiconductor memories in which at least a part of the memory is recorded, and a parking section recognition method.

  DESCRIPTION OF SYMBOLS 1 ... Parking area recognition apparatus, 2 ... Front camera, 4 ... Back camera, 5 ... CPU, 6 ... Memory, 8 ... In-vehicle LAN, 10 ... Recognition control unit, 12 ... Image acquisition part, 12 ... Behavior information acquisition part, 14 ... behavior information acquisition unit, 16 ... parking zone recognition unit, 18 ... parking zone output unit, 20 ... image processing unit, 22 ... wide area recognition unit, 24 ... narrow zone recognition unit, 30 ... target tracking unit, 32 ... tracking when moving forward Reference numeral 34: Reverse tracking unit 40: Area setting unit 50: Parking assist ECU 60: Engine ECU 70: Steering ECU 80: Brake ECU 90: HMI-ECU 100: Parking assist system

Claims (6)

An image acquisition unit that acquires a captured image of an in-vehicle camera that is mounted on a vehicle and captures the periphery of the vehicle;
A behavior information acquisition unit for acquiring behavior information indicating the behavior of the vehicle;
A detection process for detecting a candidate line that is a candidate for a parking lot line is performed from a predetermined wide-area image area in the captured image acquired by the image acquisition unit, and the pair of candidate lines detected by the detection process is included. A wide area recognition unit that recognizes each parking lot line based on a plurality of candidate lines and identifies a parking lot that includes each recognized parking lot line when the vehicle moves forward;
When the vehicle moves forward, each parking lot line constituting the parking lot specified by the wide area recognition unit is set as a target division line, and at least the position of each target division line based on the recognition result by the wide area recognition unit A forward tracking unit to track,
A backward tracking unit that tracks the position of each target lane line based on at least the tracking result of the forward tracking unit and the behavior information acquired by the behavior information acquiring unit when the vehicle is moving backward;
Based on the position of each target lane line recognized by the wide area recognition unit, an area setting unit that sets a narrow area image area smaller than at least the wide area image area in the captured image acquired by the image acquisition unit;
The detection processing is performed from the narrow area image region set by the region setting unit, and each of the targets is based on at least one of the candidate lines detected by the detection processing and a tracking result by the backward tracking unit. A narrow area recognizing unit that recognizes a lane marking and identifies the parking lane that includes each of the recognized lane markings when the vehicle is moving backward;
A parking area recognition device. The parking area recognition device according to claim 1,
The backward tracking unit takes over the tracking result of the position of each target lane line by the forward tracking unit, and is recognized by the wide area recognition unit or the narrow area recognition unit based on the captured image of the in-vehicle camera. Tracking the location of each of the target lane markings;
A parking area recognition apparatus characterized by the above. The parking area recognition device according to claim 1 or 2,
The image acquisition unit can acquire captured images of a front camera that captures the front of the vehicle and a rear camera that captures the rear of the vehicle, respectively, as the in-vehicle camera.
The forward tracking unit tracks the position of each target lane line recognized by the wide area recognition unit based on a captured image of the front camera or the rear camera,
The backward tracking unit tracks the position of each target lane line recognized by the wide area recognition unit or the narrow area recognition unit based on a captured image of the rear camera.
A parking area recognition apparatus characterized by the above. The parking area recognition device according to any one of claims 1 to 3,
The region setting unit, when the likelihood representing the likelihood that the pair of candidate lines detected by the wide area recognition unit is the parking lot line is higher than a predetermined threshold, Set the image area,
A parking area recognition apparatus characterized by the above. The parking area recognition device according to claim 4,
The region setting unit calculates the likelihood based on at least one of the length, interval, and parallelism of the pair of candidate lines.
A parking area recognition apparatus characterized by the above. An image acquisition step of acquiring a photographed image of an in-vehicle camera mounted on a vehicle and photographing the periphery of the vehicle;
A behavior information acquisition step of acquiring behavior information indicating the behavior of the vehicle;
A detection process for detecting a candidate line that is a candidate for a parking lot line from a predetermined wide-area image area in the captured image acquired by the image acquisition process is performed, and the pair of candidate lines detected by the detection process A wide area recognition step of recognizing each parking lot line based on a plurality of candidate lines and specifying a parking lot including the recognized parking lot lines when the vehicle moves forward,
When the vehicle moves forward, each parking lot line constituting the parking lot specified by the wide area recognition unit is set as a target division line, and at least the position of each target division line based on the recognition result by the wide area recognition unit Tracking process during advance tracking,
A backward tracking step of tracking the position of each target lane line based on at least the tracking result of the forward tracking step and the behavior information acquired by the behavior information acquisition step when the vehicle is moving backward;
Based on the position of each target lane line recognized by the wide area recognition step, an area setting step for setting a narrow area image region smaller than at least the wide area image area in the captured image acquired by the image acquisition step;
The target processing is performed from the narrow area image region set by the region setting step, and each of the objects is based on at least one candidate line detected by the detection processing and a tracking result by the backward tracking step. A narrow area recognition step of recognizing a lane line and identifying the parking lane that includes the recognized lane markings when the vehicle is moving backward;
A parking area recognition device.