JP2019137182A - Detection device - Google Patents

Abstract

To provide a technology for detecting an area in which a vehicle can be parked regardless of the shape of a compartment line.SOLUTION: A detection device 50 is mounted on a vehicle and comprises an acquisition unit 62 and an area detection unit 64. The area detection unit detects a plurality of candidate frames which are areas of a prescribed shape between two parallel white lines separated by a predetermined lateral distance based on a vehicle width on the basis of a captured image, and detects a smallest frame that is smallest in a distance between white lines in at least one proximity set composed of a plurality of candidate frames that have gravity center positions located within a prescribed range from each other among the plurality of candidate frames, and detects the area between the white lines in the smallest frame as a parkable area in which the vehicle can be parked based on the smallest frame.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to a technique for detecting an area mounted on a vehicle and capable of parking the vehicle.

2. Description of the Related Art Conventionally, a technique for capturing an image around a vehicle and detecting a marking line that partitions a parking frame from the captured image is known.
For example, the following Patent Document 1 proposes a technique for detecting a U-shaped double line-shaped partition line.

Japanese Patent No. 6086424

  However, in the said patent document 1, it presupposes that a division line is U-shaped double line shape. Therefore, in the said patent document 1, the problem that the area | region which can park a vehicle cannot be detected in the parking frame divided by the lane marking which is shapes other than U-shaped double-line shape, for example, a triple line may arise.

  One aspect of the present disclosure is to provide a technique for detecting an area in which a vehicle can be parked regardless of the shape of a lane marking.

  A detection device (50) according to one aspect of the present disclosure is mounted on a vehicle and includes an acquisition unit (53, S20) and a region detection unit (53, S40). An acquisition part acquires the picked-up image based on the image by which the circumference | surroundings of the vehicle containing a road surface were imaged with the at least 1 camera mounted in the vehicle. The region detection unit detects a plurality of candidate frames that are regions of a predetermined shape between two parallel white lines that are separated by a predetermined lateral distance based on the vehicle width based on the captured image. Among these, a minimum frame having the shortest distance between white lines in at least one adjacent group composed of a plurality of candidate frames each having a center of gravity within a predetermined range is detected for each adjacent group, and a white line in the minimum frame is determined based on the minimum frame. The area in between is detected as a parkable area where the vehicle can be parked.

  According to such a configuration, an area between the innermost white lines is detected as a parking area in a parking frame partitioned by a lane marking that is a shape other than the U-shaped double line shape, such as a triple white line. can do. That is, it is possible to detect the parking area regardless of the shape of the lane marking.

  Note that the reference numerals in parentheses described in this column and in the claims indicate the correspondence with the specific means described in the embodiment described later as one aspect, and the technical scope of the present disclosure It is not limited.

The block diagram which shows the structure of an imaging system and ECU. Explanatory drawing explaining the imaging range by a camera and a camera. Explanatory drawing explaining the division line and white line in a parking frame. The flowchart of a parking frame process. The flowchart of parking possible area | region process. Explanatory drawing explaining the parking frame in a bird's-eye view image. Explanatory drawing explaining the example in which a some candidate frame is produced | generated for every adjacent group. Explanatory drawing explaining the method of specifying the minimum frame from a proximity | contact group. Explanatory drawing which shows the example of a possible area | region superimposition image.

  Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following, “parallel” includes substantially parallel. That is, “parallel” is not limited to parallel in a strict sense, and may not be strictly parallel as long as the same effect can be obtained. In addition, the position described below refers to a position represented by position coordinates. The position coordinates may be represented by absolute coordinates, or may be represented by relative coordinates with respect to a predetermined position represented by absolute coordinates. As the predetermined position, for example, the center position of the bumper at the front end of the vehicle can be used.

[1. Constitution]
[1-1. overall structure]
An imaging system 1 shown in FIG. 1 is a system that is mounted on a vehicle such as a passenger car and recognizes a parking frame that will be described later. The imaging system 1 includes an electronic control unit (hereinafter, ECU) 50. Hereinafter, the vehicle on which the imaging system 1 is mounted is also referred to as the host vehicle. The imaging system 1 may include a camera 10, a sensor group 20, a display control device 30, and a vehicle control device 40.

  The camera 10 includes a front camera 11, a left side camera 12, a right side camera 13, and a rear camera 14. Each of the cameras 11 to 14 is a well-known camera configured using a CCD image sensor or a CMOS image sensor. As shown in FIG. 2, the front camera 11 is installed, for example, on a bumper at the front end of the vehicle so that the road surface in front of the vehicle is within the shooting range 101. The left side camera 12 is installed, for example, on the left side mirror so that the road surface on the left side of the vehicle becomes the shooting range 102.

  The right side camera 13 is installed, for example, on the right side mirror so that the road surface on the right side of the vehicle becomes the photographing range 103. The rear camera 14 is installed, for example, on a bumper at the rear end of the vehicle so that the road surface behind the vehicle becomes the photographing range 104. Each of the cameras 11 to 14 repeatedly shoots at a preset time interval, for example, tens of seconds (seconds) to tens of seconds (seconds), and outputs the shot images (hereinafter referred to as camera images) to the ECU 50. To do. The camera image is an image captured by the camera 10, that is, each of the cameras 11 to 14, and is an image captured around the vehicle 9 including the road surface.

  The sensor group 20 is a sensor that measures the behavior of the vehicle 9. Specifically, the sensor group 20 is a plurality of sensors including a vehicle speed sensor 21 that measures the vehicle speed of the vehicle 9 and a steering angle sensor 22 that measures the steering angle of the vehicle 9.

  The display control device 30 includes a well-known microcomputer including a CPU, a ROM, a RAM, and a semiconductor memory such as a flash memory. The display control device 30 converts the image output from the ECU 50 into a video signal that can be displayed on a display device (not shown), and outputs the video signal to the display device. The display device is a device for displaying an image, and is disposed in the passenger compartment. The display device includes, for example, a liquid crystal display.

  The vehicle control device 40 includes a well-known microcomputer including a CPU, a ROM, a RAM, and a semiconductor memory such as a flash memory. The vehicle control device 40 acquires the position of a parking area that will be described later, which is output from the ECU 50, generates a track of the host vehicle for parking in the parking area, and moves the host vehicle according to the track. Controls acceleration / deceleration of the host vehicle, steering angle of the host vehicle, and the like.

  The ECU 50 includes a microcomputer having a CPU and a semiconductor memory such as a ROM, a RAM, and a flash memory. Each function realized by the ECU 50 is realized by the CPU executing a program stored in a non-transitional physical recording medium. In this example, the semiconductor memory corresponds to a non-transitional tangible recording medium storing a program. Also, by executing this program, a method corresponding to the program is executed. The ECU 50 may include a single microcomputer or a plurality of microcomputers.

  As shown in FIG. 1, the ECU 50 includes an imaging signal input unit 51, a vehicle signal input unit 52, and an image processing unit 53. The image processing unit 53 includes an input processing unit 61, an acquisition unit 62, a white line recognition unit 63, a region detection unit 64, and an output unit 65. The method for realizing the functions of these units is not limited to software, and some or all of the functions may be realized using one or a plurality of hardware. For example, when the function is realized by an electronic circuit that is hardware, the electronic circuit may be realized by a digital circuit, an analog circuit, or a combination thereof.

  The photographing signal input unit 51 acquires camera images picked up by the cameras 11 to 14 and supplies them to the image processing unit 53. The vehicle signal input unit 52 acquires the vehicle speed and the steering angle from the sensor group 20 and supplies them to the image processing unit 53. The image processing unit 53 performs parking frame processing, which will be described later.

[1-2. Parking frame]
Next, the parking frame will be described. The parking frame here is an area for parking a vehicle in a parking lot. The parking frame is partitioned by a lane marking that divides an area where the vehicle is parked. Here, the two lane markings and the area between the two lane markings are called parking frames. The lane marking is a line for representing a parking frame, and is composed of a white line or a yellow line drawn on the road surface of the parking lot. In the following, for example, the lines constituting the division lines including the lines of colors other than white such as yellow lines are referred to as white lines. The lane marking is composed of at least one white line having a width within a preset range. That is, the lane marking may be composed of a single white line or may be composed of a plurality of white lines.

  One parking frame is partitioned by at least two partition lines that are parallel to each other and separated by a lateral distance that is a length corresponding to the width of the vehicle. A vehicle here represents a general passenger car. That is, the lateral distance is a length within a predetermined range including the vehicle width W of a general passenger car. The vehicle width W is the length in the short direction of the vehicle. If the vehicle width W is expressed by α> lateral distance> vehicle width W> β, α to β correspond to a predetermined range.

  FIG. 3 illustrates a parking frame 71 partitioned by a lane line 81 composed of three white lines 91 to 93 and a lane line 82 composed of two white lines 94 to 95. Moreover, the parking frame 72 partitioned by the lane marking 82 consisting of the two white lines 94-95 and the lane marking 83 consisting of the three white lines 96-98 is illustrated.

  When the vehicle is parked in the parking frame, it is desirable that the vehicle is parked inside the lane marking. The inside here means the side facing the vehicle when the vehicle is parked in the parking frame. That is, it is desirable that each of the two parallel lane markings is parked in a region between the white lines located on the innermost side among the white lines constituting the lane marking. For example, in the case of the parking frame 71 described above, it is desirable to park in a region sandwiched between the white line 93 and the white line 94 that are the innermost white lines among the white lines constituting the lane lines 81 and 82. .

  The parking possible area mentioned below is an area where the vehicle can be parked. That is, when the vehicle is parked in the parking frame, it is desirable that the area between the white lines located on the innermost side among the white lines constituting the lane markings is detected as a parking area. Below, the parking frame process which is a process for detecting such a parking possible area | region is demonstrated.

[2. processing]
[2-1. Parking frame processing]
Next, parking frame processing executed by the image processing unit 53 will be described with reference to the flowchart of FIG.

  In S <b> 10, the image processing unit 53 acquires a camera image via the shooting signal input unit 51. The image processing unit 53 converts the acquired camera image into a digital signal by sampling.

  In S <b> 20, the image processing unit 53 converts the camera images converted into digital signals from the four cameras 11 to 14 into a bird's-eye view seen from a preset virtual viewpoint, and synthesizes the bird's-eye view showing the surroundings of the vehicle 9. Generate an image. In particular, in this embodiment, a bird's-eye view image in which the periphery of the vehicle 9 is looked down from the vertical direction is generated. In the present embodiment, the bird's eye view image corresponds to the captured image.

  In S30, the image processing unit 53 recognizes a white line from the bird's eye view image generated in S20. For example, the image processing unit 53 detects a feature point representing a brighter part than the road surface from the generated bird's eye view image. In the present embodiment, the image processing unit 53 uses the edge point of the region whose luminance value is higher than the threshold value by more than the surrounding as the feature point, and detects the feature point by applying a Sobel filter or the like to the bird's eye view image. Then, the image processing unit 53 detects a straight line by applying Hough transform or the like to the detected feature point.

  The white line has a width within a predetermined range (hereinafter, line width). In the present embodiment, the image processing unit 53 detects two straight lines that are separated from the above-described line widths among the detected straight lines as straight lines representing the edges of the white line. The edge of the white line is the boundary between the white line and the road surface. Here, the image processing unit 53 detects a straight line passing through the center of two straight lines representing the edge of the white line as a straight line representing the white line. The image processing unit 53 recognizes a white line by regarding the straight line representing the white line as a white line.

In S40, the image processing unit 53 detects a parking area based on the white line recognized in S30. Details of the parking area detection will be described later.
In S50, the image processing unit 53 performs output to the vehicle control device 40 and the display control device 30 based on the available parking area detected in S40.

  Specifically, the image processing unit 53 tracks the parking area, recognizes the position of the parking area, and outputs information representing the position of the parking area to the vehicle control device 40. Tracking refers to processing for tracking an object in a captured image by associating the amount of movement of the host vehicle 9 with the amount of movement of the object in the captured image. In the present embodiment, the captured image referred to here corresponds to the above-described bird's-eye view image.

  Thereby, the vehicle control device 40 generates, for example, a track of the host vehicle for parking in the parkable region based on the information indicating the position of the parkable region, and moves the host vehicle according to the track. Controls acceleration / deceleration of the host vehicle, steering angle of the host vehicle, and the like.

  In addition, the image processing unit 53 generates an image (hereinafter referred to as a possible area superimposed image) in which the parking area is superimposed on the above-described bird's-eye view image while taking into account the movement amount of the host vehicle, and the possible area superimposed image is displayed on the display control device. Output to 30. Thereby, the display control device 30 causes the display device to display the possible region superimposed image. The possible region superimposed image is represented as an image in which the amount of movement of the host vehicle is taken into account as the host vehicle moves.

The image processing unit 53 ends the parking frame process.
In the present embodiment, the process of S10 corresponds to the process of realizing the function as the input processing unit 61, the process of S20 is equivalent to the process of realizing the function as the acquisition unit 62, and the process of S30 is white line recognition. The process of S40 corresponds to the process of realizing the function as the area detector 64, the process of S40 corresponds to the process of realizing the function as the area detector 64, and the process of S50 corresponds to the process of realizing the function as the output unit 65.

[2-2. Parking area processing]
Next, the parking possible area | region process which the image process part 53 performs in S40 is demonstrated using the flowchart of FIG.

  First, in S110 to S130, the image processing unit 53 identifies two parallel white lines that are separated from each other among the white lines recognized in S30. Specifically, in S110, the image processing unit 53 selects two lines from the white lines recognized in S30.

  Next, in S120, the image processing unit 53 determines whether the two white lines are parallel. The image processing unit 53 shifts the process to S130 when the two white lines are parallel, and shifts the process to S150 when they are not parallel.

  Subsequently, in S130, the image processing unit 53 determines whether or not the distance between the two parallel white lines is a predetermined lateral distance. As described above, the lateral distance is a value within the range of α to β. That is, the image processing unit 53 determines that the lateral distance is separated when the distance between the two parallel white lines is within the range of the lateral distance, that is, within the range of α to β described above. The image processing unit 53 shifts the process to S140 when the distance between the two parallel white lines is separated from the horizontal distance, and shifts the process to S150 when the distance is not separated.

  As described above, when two parallel white lines that are separated from each other are identified from the white lines recognized in S30, the process proceeds to S140. Hereinafter, two parallel white lines that are separated from each other by a lateral distance are also referred to as the above-described two parallel white lines.

  In S140, the image processing unit 53 generates a candidate frame. The candidate frame is a region having a predetermined shape (hereinafter, a predetermined shape) between the two parallel white lines described above. In the present embodiment, the predetermined shape is a quadrangular shape. Here, the quadrangle may include a parallelogram, a rectangle, and a square. The image processing unit 53 generates a rectangular region having a predetermined width (hereinafter referred to as a detection width) between the two parallel white lines described above as a candidate frame. In addition, the image processing unit 53 sets a virtual line connecting the ends of the two parallel white lines included in the bird's eye view image as a boundary line that separates the inside and the outside of the rectangular area.

In other words, the image processing unit 53 generates a quadrangular area between two parallel white lines that are separated from each other by a lateral distance and has a detection width from the virtual line as a candidate frame.
The detection width may be equal to or less than the general vehicle length that is the length in the longitudinal direction of a general passenger car. In the present embodiment, the detection width is assumed to be 1/2 of the general vehicle length. However, the detection width is not limited to this, and the detection width is set to an arbitrary value depending on how much the white line captured in the bird's eye view image is reflected from the end of the white line. Can be set.

  The image processing unit 53 calculates the barycentric position of the generated candidate frame, sets a candidate flag, and causes the process to proceed to S160. The gravity center position and the candidate flag are stored in a memory provided in the ECU 50. As will be described later, the candidate flag is a flag used to specify a candidate frame that represents a parking area from among a plurality of candidate frames.

In S150, the image processing unit 53 does not generate a candidate frame and shifts the process to S160.
In S160, the image processing unit 53 determines whether all combinations have been confirmed in selecting two lines from the white lines recognized in S30. Here, the image processing unit 53 shifts the process to S110 when all the combinations have not been confirmed, and repeats the processes of S110 to S160. The image processing unit 53 shifts the processing to S170 when all the combinations have been confirmed.

  Here, with reference to FIGS. 6 and 7, examples of candidate frames generated in a situation where the parking frames 71 to 72 are positioned on the left side with respect to the arrow A that is the traveling direction of the vehicle 9 as shown in FIG. 3. explain. In FIG. 6, a candidate frame 304 is shown as an example of a rectangular area between the two parallel white lines described above and having a detection width K from the virtual line 110 described above. Each of the plurality of candidate frames is included in the same strip-shaped region parallel to the imaginary line 110 and having the above-described detection width K as a lateral width. In FIG. 7, the plurality of candidate areas are shown side by side.

  For example, as shown in FIG. 6, if a bird's-eye view image includes a plurality of parking frames, or if the lane marking consists of a plurality of white lines, a plurality of candidate frames 201 to 206, as shown in FIG. 7, 301 to 306 are generated. For example, although not shown, when one bird's-eye view image includes one parking frame and the partition line of the parking frame is composed of one white line, one candidate frame is generated. The center-of-gravity position and the candidate flag are stored for each of the one or more candidate frames. In FIG. 7, the center of gravity position in the candidate frame is indicated by a black circle.

  FIG. 6 is an example of a bird's-eye view image generated based on the camera image in the situation of FIG. 3, but the bird's-eye view image includes a part of the parking frames 71 to 72. The bird's eye view image generated in this way does not necessarily include the entire parking frames 71 to 72.

  However, in the present embodiment, the bird's eye view image includes an end portion on the side close to the host vehicle 9 among the end portions in the longitudinal direction of the parking frames 71 to 72. That is, the bird's-eye view image includes an end portion on the side close to the host vehicle 9 among the end portions in the longitudinal direction of the white lines constituting the lane markings. A line connecting the end portions on the two parallel white lines described above corresponds to the virtual line 110.

  In S170, the image processing unit 53 determines whether a plurality of candidate frames have been generated. Here, when a plurality of candidate frames are generated, the image processing unit 53 shifts the processing to S180. On the other hand, when one candidate frame is generated, the image processing unit 53 identifies the one candidate frame as a minimum frame which will be described later, and shifts the processing to S230.

  In S180 to S220, the image processing unit 53 specifies the minimum frame for each proximity group from at least one adjacent group. The adjacent group is a plurality of candidate frames whose center-of-gravity positions are located within a predetermined range among the plurality of candidate frames. That is, the proximity set is a group including at least two candidate frames. The minimum frame is a candidate frame in which the distance between the two parallel white lines described above in the adjacent group is the smallest. However, when only one candidate frame is generated based on the bird's eye view image, the minimum frame means one generated candidate frame.

Specifically, the image processing unit 53 selects two candidate frames from a plurality of candidate frames in S180.
In S190, the image processing unit 53 determines whether candidate flags are set in both of the two selected candidate frames. The image processing unit 53 shifts the process to S200 when the candidate flag is set in both, and shifts the process to S220 when the candidate flag is set only in one.

  In S200, the image processing unit 53 determines whether or not the center of gravity position is within a predetermined range for the two selected candidate frames. Specifically, the image processing unit 53 determines that the centroid position is within a predetermined range when the distance between the centroid positions (hereinafter, the distance between the centroids) is less than a predetermined proximity threshold. The image processing unit 53 shifts the process to S210 if the gravity center position is within the predetermined range, and shifts the process to S220 if the gravity center position is not within the predetermined range.

  The proximity threshold value can be set to, for example, a value representing the width of a general passenger car, a value representing the width of a lane marking, or the like. For example, as shown in FIG. 6 and FIG. 7, the proximity threshold values are the candidate frames 201 to 206 in the parking frame 71 and the candidate frames 301 to 306 in the parking frame 72 for the adjacent parking frames 71 and 72 in the bird's eye view image 501. As long as the values can be grouped into different adjacent groups.

  In S210, the image processing unit 53 detects the distance between the two parallel white lines described above for each of the two candidate frames. Then, the image processing unit 53 resets the candidate flag for the candidate frame having the larger distance between the two parallel white lines described above among the two candidate frames.

  In S220, the image processing unit 53 confirms all combinations in selecting two candidate frames in S180 from all candidate frames generated when an affirmative determination is made in S160. Judging. Here, the image processing unit 53 shifts the processing to S180 when not all the combinations have been confirmed, and repeats the processing of S180 to S220. The image processing unit 53 shifts the processing to S230 when all the combinations have been confirmed.

  As a result of the processing of S180 to S220, as shown in FIG. 8, in the candidate frames 301 to 306 whose center of gravity is within a predetermined range, for example, two candidate frames selected in the order of A1 to A5, respectively, The distances between the parallel white lines of the books are compared, and the candidate flags are sequentially reset based on the comparison result. As a result, only the candidate flag of the candidate frame having the shortest distance between the above-described two parallel white lines among the plurality of candidate frames included in the same adjacent group remains in a state where it is set.

  That is, in the example of FIG. 7, only the candidate flag of the candidate frame 204 is set and the candidate flags of the other candidate frames are reset in the adjacent group including the candidate frames 201 to 206. Further, in the proximity group including the candidate frames 301 to 306, only the candidate flag of the candidate frame 304 is set, and the candidate flags of other candidate frames are reset. Note that the candidate frames 201 to 206 whose center-of-gravity positions are within a predetermined range form a proximity set for the parking frame 71. In addition, candidate frames 301 to 306 whose gravity center positions are within a predetermined range constitute a proximity set for the parking frame 72.

  In S230, the image processing unit 53 identifies a minimum frame and detects a parking area based on the minimum frame. First, the image processing unit 53 detects a candidate frame in which the candidate flag is set as the minimum frame in this step that is shifted to when the determination is affirmative in S220. In the above-described example, the candidate frame 204 is detected as the minimum frame in the adjacent group including the candidate frames 201 to 206, and the candidate frame 304 is detected as the minimum frame in the adjacent group including the candidate frames 301 to 306.

  Next, the image processing unit 53 detects an area between white lines in the minimum frame as a parking area. That is, for the proximity group including the candidate frames 201 to 206, in other words, for the parking frame 71, the area between the white line 93 and the white line 94 is detected as the parking area 401. In addition, for the proximity group including the candidate frames 301 to 306, in other words, for the parking frame 72, the area between the white line 95 and the white line 96 is detected as the parking area 402.

  In the example of FIG. 7, the bird's-eye view image 501 includes only a part of the parking frame, and all the areas between the white lines in the minimum frame that are reflected in the bird's-eye view image 501 are detected as parking areas. Is done.

  The image processing unit 53 stores information representing the position of the parking area detected in this way in a memory included in the ECU 50. For example, the positions of the four corners of the rectangular parking area may be stored as information representing the parking area. The image processing unit 53 ends the process.

Thus, for example, when information indicating the position of the parking area is output to the display control device 30, a possible area superimposed image 502 as shown in FIG. 9 is displayed on the display device.
[3. effect]
According to the embodiment described in detail above, the following effects can be obtained.

  (3a) In the ECU 50, the image processing unit 53 acquires an image captured by the camera 10. The image processing unit 53 detects a plurality of candidate frames based on the captured image, and white lines in at least one adjacent set including a plurality of candidate frames whose centroid positions are within a predetermined range among the plurality of candidate frames. The smallest frame with the smallest distance is detected for each adjacent group, and the area between the white lines in the smallest frame is detected as a parking area.

  As a result, the area between the innermost white lines can be detected as a parking area in a parking frame that is partitioned by a lane marking having a shape other than the U-shaped double line shape, such as a triple white line. That is, it is possible to detect the parking area regardless of the shape of the lane marking.

(3b) The image processing unit 53 calculates a center-of-gravity position in the candidate frame using a quadrangular region having a predetermined detection width as a candidate frame.
As a result, even if not all of the white lines that demarcate the parking frame are captured in the captured image, it is possible to identify the parking area if at least the white lines corresponding to the detection width are captured.

  (3c) The image processing unit 53 is the above-described quadrangular region, and the virtual line connecting the predetermined positions in the longitudinal direction on the two parallel white lines is bounded between the inside and outside of the quadrangular region. Using the region to be a line as a candidate frame, the position of the center of gravity in the candidate frame is calculated.

As a result, even if not all of the white lines are shown in the captured image, it is possible to specify the parking area based on the quadrangular area having the virtual line as the boundary line.
(3d) It is assumed that the predetermined position in the longitudinal direction on the two white lines is the end of the two white lines included in the bird's-eye view image as the captured image.

As a result, even if not all of the white line is captured in the captured image, the parking area can be specified as long as at least the end of the white line is captured.
(3e) The image processing unit 53 detects the minimum frame for each adjacent group in the plurality of adjacent groups, and detects a parking area for each minimum frame.

As a result, it is possible to detect parking areas in a plurality of adjacent parking frames.
(3f) The image processing unit 53 acquires, as a captured image, a bird's eye view image obtained by converting a camera image captured by the camera 10 into an image from a viewpoint above the vehicle 9. The image processing unit 53 detects a parking area based on the bird's eye view image.

As a result, the image processing unit 53 can easily generate a possible region superimposed image using the acquired bird's-eye view image.
[4. Other Embodiments]
As mentioned above, although embodiment of this indication was described, this indication is not limited to the above-mentioned embodiment, and can carry out various modifications.

  (4a) In the above embodiment, a candidate frame that is a quadrangular region is detected. However, the present invention is not limited to this. For example, an area having an arbitrary predetermined shape other than a square, such as a circle or an ellipse, may be detected as a candidate frame. For example, a linear region may be detected as a candidate frame. In this case, the image processing unit 53 may specify a plurality of candidate frames included in the proximity group based on the center position using the center position of the linear region instead of the center of gravity position.

  (4b) In the above-described embodiment, the above-described two parallel white lines are defined by using the virtual line connecting the ends of the above-described two parallel white lines included in the bird's-eye view image as the captured image as the boundary line of the candidate frame. A rectangular area having a detection width from the boundary line is detected as a candidate frame, but the present invention is not limited to this. The image processing unit 53 may set a virtual line connecting arbitrary positions determined in advance in the longitudinal direction of the two white lines as a boundary line of the candidate frame. Then, the image processing unit 53 may detect a rectangular area having a detection width from the virtual line, which is an area between the two parallel white lines described above, as a candidate frame.

As a result, if a part of the white line is captured in the captured image, the parking area can be specified as in (3a) above.
Moreover, when all the parking frames are included in the bird's eye view image as the captured image, the image processing unit 53 may detect all the regions sandwiched between the two parallel white lines described above as candidate frames.

  In addition, when the bird's-eye view image as the captured image includes all of the parking frame, the image processing unit 53 is a region sandwiched between the two parallel white lines described above, and uses the general vehicle length as the detection width. A rectangular area may be detected as a candidate frame.

  Further, the image processing unit 53 is an area between the two parallel white lines described above when both of the parking frames are included in the bird's eye view image as the captured image, and both ends of the two parallel white lines. A quadrangular region having two imaginary lines connecting the end portions of the two as boundary lines may be detected as a candidate frame.

  (4c) In the above embodiment, the camera 10 includes the front camera 11, the left side camera 12, the right side camera 13, and the rear camera 14. However, the present invention is not limited to this. The camera 10 may include at least one of the front camera 11, the left side camera 12, the right side camera 13, and the rear camera 14.

  Moreover, in the said embodiment, although the bird's-eye view image was produced | generated based on all the camera images by the front camera 11, the left side camera 12, the right side camera 13, and the rear camera 14, it is not limited to this. . For example, even if a bird's-eye view image is generated based on at least one camera image among the front camera 11, the left side camera 12, the right side camera 13, and the rear camera 14 according to the operation of parking the host vehicle in the parking frame. Good.

  For example, if the host vehicle 9 is retracted and parked in the parking frame 72 shown in FIG. 3, among the plurality of cameras 11 to 14 for each operation of the vehicle 9 during parking as shown by arrows A to C. At least one camera may be selected, a bird's eye view image may be generated based on a camera image acquired by the selected camera, and a parking area may be detected. An arrow A indicates an operation of moving forward to detect a parking area. An arrow B indicates a preparatory operation before retreat performed while tracking the parking area. An arrow C indicates an operation until the vehicle moves backward and stops on the parking frame 72.

  (4d) In the above embodiment, a bird's eye view image in which the periphery of the vehicle 9 is viewed from the vertical direction is generated, but the present invention is not limited to this. A bird's-eye view image viewed from an arbitrary virtual viewpoint set in advance may be generated.

  (4e) In the above-described embodiment, an example in which the tip is squarely closed in a lane marking composed of a plurality of white lines is shown, but the present invention is not limited to this. The lane marking composed of a plurality of white lines may not be closed at the tip. Moreover, the tip of the lane marking composed of a plurality of white lines may be closed in a U shape. Moreover, the white line which comprises a division line should just be one or more arbitrary numbers.

  (4f) In the above embodiment, the image processing unit 53 is configured to cause the display control device 30 to display the bird's eye view image generated by the image processing unit 53 on the display device. However, the present invention is not limited to this. is not. For example, the image processing unit 53 may be configured to display the camera image itself captured by the cameras 11 to 14 on the display control device 30. The image processing unit 53 may be configured to cause the display control device 30 to display an image in which a parking frame is superimposed on the camera image.

  (4g) A plurality of functions of one constituent element in the above embodiment may be realized by a plurality of constituent elements, or a single function of one constituent element may be realized by a plurality of constituent elements. . Further, a plurality of functions possessed by a plurality of constituent elements may be realized by one constituent element, or one function realized by a plurality of constituent elements may be realized by one constituent element. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified from the wording described in the claims are embodiments of the present disclosure.

  (4h) The above-described ECU 50, the CPU provided in the ECU 50, the imaging system 1, a program for causing the ECU 50 to function, a non-transition actual recording medium such as a semiconductor memory in which the program is recorded, a parking area detection method, and the like The present disclosure can also be realized in the form of.

  In the above embodiment, the ECU 50 corresponds to a detection device, and the image processing unit 53 corresponds to an acquisition unit and a region detection unit. Further, S20 corresponds to processing as an acquisition unit, and S40 corresponds to processing as an area detection unit.

  10 camera, 50 ECU, 53 image processing unit.

Claims (6)

A detection device (50) mounted on a vehicle,
An acquisition unit (53, S20) for acquiring a captured image based on an image of the surroundings of the vehicle including a road surface by at least one camera (10) mounted on the vehicle;
Based on the captured image, a plurality of candidate frames that are regions of a predetermined shape between two parallel white lines separated by a predetermined lateral distance based on a vehicle width are detected, and a center of gravity position among the plurality of candidate frames Detects the smallest frame having the smallest distance between the white lines for each of the neighboring groups in at least one neighboring group composed of a plurality of candidate frames positioned within a predetermined range, and based on the smallest frame, An area detection unit (53, S40) for detecting an area between the white lines as a parkable area where the vehicle can be parked;
A detection device comprising: The detection device according to claim 1,
The candidate frame is a rectangular area;
The said area | region detection part is a square area | region, Comprising: The area | region of a predetermined width is used as the said candidate frame, The detection apparatus calculates the gravity center position in the said candidate frame. The detection device according to claim 1 or 2,
The candidate frame is a rectangular area;
The area detection unit is the quadrangular area, wherein the quadrangular area has a virtual line connecting predetermined positions in the longitudinal direction on the two parallel white lines as a boundary line inside and outside the area. A center of gravity position in the candidate frame. The detection device according to claim 3,
The predetermined position in the longitudinal direction on the two white lines is an end of the two white lines included in the captured image. The detection device according to any one of claims 1 to 4,
The said area | region detection part detects the said minimum frame in every said adjacent group in the said some adjacent group, and detects the said parking possible area for every said minimum frame based on the said minimum frame. The detection device according to any one of claims 1 to 5,
The acquisition unit acquires, as the captured image, a bird's eye view image obtained by converting an image captured by the camera into an image from an upper viewpoint of the vehicle,
The said area | region detection part detects the said parking possible area | region based on the said bird's-eye view image.